This is Part 2 of a five-part series on the internal mechanics of large language models. Part 1 covered tokenization, embeddings, and positional encoding. Part 2 derives the self-attention mechanism from first principles, covers multi-head attention, and examines the KV cache. The Problem Attention Solves At the end of Part 1, we had a matrix X∈Rn×dX \in \mathbb{R}^{n \times d}, which was a one ddd-dimensional vector per token, encoding both semantic identity and position. The first transformer layer receives this matrix and must do something critical: allow each token to incorporate information from every other token in the sequence before passing its updated representation to the feed-forward network. This is…