-
How Large Language Models Work — Part 3: The Transformer Block and Architecture
This is Part 3 of a five-part series on the internal mechanics of large language models. Part 2 derived scaled dot-product attention and the KV cache. Part 3 traces the complete transformer block consisting of feed-forward networks, layer normalisation, and residual streams; it examines the architectural differences between decoder-only and encoder-decoder models. From Attention Output to Transformer Block At the end of Part 2, self-attention had produced an output matrix O∈Rn×dO \in \mathbb{R}^{n \times d} or one updated vector per token, now containing contextual information drawn from every other position in the sequence. But this is only the first half of a single transformer block. Before the signal moves to…