-
How Large Language Models Work — Part 5: Inference, Sampling, Emergent Behaviour, and Open Frontiers
This is the final part of a five-part series on the internal mechanics of large language models. Parts 1–4 covered tokenization, embeddings, attention, the transformer block, and the training and alignment pipeline. Part 5 addresses inference: how a trained model generates text, the mathematics of sampling, context window management, emergent capabilities, and the open research frontiers shaping the next generation of LLMs. From Trained Weights to Generated Text A trained, aligned LLM is a fixed mathematical function: given a sequence of input tokens, it produces a probability distribution over the next token. Inference is the process of repeatedly applying this function to generate text, while the engineering decisions made at…