Attention and the Transformer Block, Derived From an Average

From 1997 to about 2017 the summary was a recurrent state: keep a vector $\mathbf{h}$ and update it at every token, $\mathbf{h}_t = f(\mathbf{h}_{t-1}, \mathbf{x}_t)$. Fixed size, variable input, exactly the right shape.

It failed in three ways, and this chapter's architecture answers all three. Backpropagating through $T$ steps multiplies $T$ Jacobians, so the gradient vanishes or explodes — the disease Chapter 5 measured inside a single $\tanh$ node. The LSTM¹ was designed against exactly that and pushed the usable range from tens of steps to hundreds, without changing the fact that information from token 5 reaches token 500 only by surviving 495 sequential updates. The whole source had to fit in one vector: in sequence-to-sequence translation² an encoder compresses the input into its final state. Bahdanau, Cho and Bengio named that bottleneck and fixed it in 2014, three years before the transformer, by letting the decoder take a weighted sum of all the encoder states with weights it computed itself.³ Everything below is that idea, applied by a sequence to itself, with the recurrence deleted. And the update is sequential by construction: $\mathbf{h}_t$ needs $\mathbf{h}_{t-1}$, and a GPU with ten thousand cores can do nothing with that. The architecture that won is not obviously smarter; it is the one whose expensive step is a matrix multiply.

The other classical inductive bias, convolution — slide one small filter over the whole input, so a feature detected anywhere is detected everywhere — is not built here either; it is nearly exactly right for images and is delegated to a vision course. Neither recurrence nor convolution reappears after this page, which is why neither gets a chapter: Chapter 1 promised the omissions would be declared rather than quiet.

The most obvious function of a variable number of vectors that returns one vector is the average:

Any number of inputs, fixed output size, differentiable, free. Embedding table plus this average plus a linear layer to the vocabulary is a complete language model in fifteen lines. It is also terrible, and how it is terrible is the whole derivation.

The corpus below is one megabyte of Shakespeare, 1,115,394 characters, through a byte-level BPE tokenizer of the kind built in Chapter 7 with a vocabulary of 1024: 459,760 tokens at 2.43 characters each, split 90/10. Every model is 128 wide, sees 128 tokens, and trains for 3000 steps of AdamW at $10^{-3}$ with a batch of 64. Perplexity is on the held-out split.⁴

Read the second row twice. Averaging the context does not help a little; it makes the model four times worse than ignoring the context entirely. Two reasons, both provable rather than empirical.

The average cannot see order. Addition commutes, so shuffling the window leaves the summary unchanged — not approximately:

A = torch.tril(torch.ones(T, T))
A = A / A.sum(1, keepdim=True)          # rows of the averaging matrix
y = x[torch.randperm(T)]                # the same tokens, shuffled
print((A[-1] @ x - A[-1] @ y).abs().max().item())

2.9802322387695312e-08

Floating-point noise on a reordered sum: the two summaries are the same vector. A model whose only view of the context is an average cannot distinguish the dog bit the man from the man bit the dog. Row three proves this is not fixable by adding positions to the inputs — a learned position embedding on every token before averaging bought 2.14 points out of 188. The positions go into the sum, and the sum forgets them.

And the average drowns the present. At position 100 the current token is one hundredth of the summary. That has a cheap fix you already own: keep the token and add the summary to it — a residual connection, from Chapter 6, and row four is what it does. With the dilution repaired, the uniform average contributes nothing at all: 60.45 against a baseline of 59.71. Every token is in there, weighted equally, and equal weighting is the same as no information.

The problem is not the averaging. It is the weights.

Averaging over a growing prefix looks like a loop. It is one multiplication by a lower-triangular matrix whose rows sum to one — and also, exactly, a softmax:

loop = torch.stack([x[:t + 1].mean(0) for t in range(T)])   # the obvious version

A = torch.tril(torch.ones(T, T))
A = A / A.sum(1, keepdim=True)
mat = A @ x                                                  # the same thing

S = torch.zeros(T, T).masked_fill(torch.tril(torch.ones(T, T)) == 0, float("-inf"))
soft = F.softmax(S, dim=-1) @ x                              # and the same thing again

loop vs matmul   max |diff| = 5.960464477539063e-08
loop vs softmax  max |diff| = 5.960464477539063e-08

the averaging matrix A (rows sum to 1, upper triangle is zero):
  1.000 0.000 0.000 0.000 0.000 0.000
  0.500 0.500 0.000 0.000 0.000 0.000
  0.333 0.333 0.333 0.000 0.000 0.000
  0.250 0.250 0.250 0.250 0.000 0.000
  0.200 0.200 0.200 0.200 0.200 0.000
  0.167 0.167 0.167 0.167 0.167 0.167

Three named components of a transformer are now on screen. The triangle is the causal mask, forced by the objective: if position $t$ could see position $t{+}1$ the answer would be in the input — the leak Chapter 6 told you to audit for, except inside the architecture. The softmax is how the mask is implemented: setting forbidden entries to $-\infty$ sends them to exactly zero and normalises what remains, so masking and normalising are one operation. (Use $-\infty$, not -1e9: it is the value the masking means, it survives a cast to float16 as $-\infty$, and it spares you deciding whether the constant you picked is large enough for the range you happen to be in — which is Chapter 2's floating-point box asking a question you do not have to answer.) And the scores are the free parameter. The uniform average is what you get when every allowed score is the same number; put any numbers there and the softmax turns them into valid weights.

The rest of this chapter is one question: where do those numbers come from?

They cannot be plain parameters. A learned $T \times T$ matrix would be identical for every sentence — it could encode "look four tokens back" but never "look at the noun this pronoun refers to". The weight linking position $t$ to position $i$ must depend on what is at both positions, because relevance is a relation, not a property: the word it is not intrinsically relevant, it is relevant to something.

The cheapest function of two vectors returning a number is the dot product of Chapter 1. Score position $i$ for position $t$ as $\mathbf{x}_t \cdot \mathbf{x}_i$ and the mechanism works — badly, in two ways that force everything else. A vector's dot product with itself is its squared norm, so every token would attend mostly to itself. And the relation would be symmetric: if it attends strongly to animal, then animal attends strongly to it, which is false about language, where an adjective needs its noun far more than the noun needs the adjective.

So give each token two roles, as two learned linear maps of it: what this position is looking for, $\mathbf{q}_t = W_q\mathbf{x}_t$, the query; and what it offers to be found by, $\mathbf{k}_i = W_k\mathbf{x}_i$, the key. Score $\mathbf{q}_t \cdot \mathbf{k}_i$ and the symmetry is gone, because $W_q \neq W_k$: a token can advertise one thing and search for another.

One thing is still wrong. The weighted sum was over the $\mathbf{x}_i$ themselves, which forces the thing that gets copied to be the thing that gets matched. Matching wants the features that identify a token; copying wants the features that are useful downstream. So learn a third map, $\mathbf{v}_i = W_v\mathbf{x}_i$, the value, and sum those.

The formula is now bookkeeping:

with $M$ the causal mask, zero on and below the diagonal and $-\infty$ above. In code it is thirty lines, twenty of which are shapes:

class Head(nn.Module):
    """One head of causal self-attention."""

    def __init__(self, d_model, d_head, block):
        super().__init__()
        self.q = nn.Linear(d_model, d_head, bias=False)      
        self.k = nn.Linear(d_model, d_head, bias=False)      
        self.v = nn.Linear(d_model, d_head, bias=False)      
        self.d_head = d_head
        self.register_buffer("mask", torch.tril(torch.ones(block, block)).bool())

    def forward(self, x):
        T = x.shape[1]
        q, k, v = self.q(x), self.k(x), self.v(x)
        s = q @ k.transpose(-2, -1) / math.sqrt(self.d_head)          
        s = s.masked_fill(~self.mask[:T, :T], float("-inf"))          
        w = F.softmax(s, dim=-1)                                      
        return w @ v                                                  

Score, mask, normalise, mix. Everything else is a projection.

Almost every explanation of $\sqrt{d_k}$ says "to keep the softmax from saturating", which is true and explains nothing. The argument is two lines of the variance from Chapter 2. If the entries of $\mathbf{q}$ and $\mathbf{k}$ are independent with mean zero and variance one, each product $q_j k_j$ has variance one, and variances of independent things add:

So the scores have standard deviation $\sqrt{d_k}$. Measured over twenty thousand random pairs:

     d     Var(q.k)         std   sqrt(d)
     4        3.975       1.994     2.000
    16       16.071       4.009     4.000
    64       64.249       8.016     8.000
   256      253.065      15.908    16.000
  1024     1015.562      31.868    32.000

Why that matters: the softmax is scale-sensitive in a way a linear layer is not. Doubling a linear layer's input doubles its output; multiplying scores by ten before a softmax turns a soft blend into a hard choice. One row of 64 scores, with and without the division:

"Effective tokens" is the exponential of the entropy: how many positions the row really averages over. Undivided, at $d_k = 256$, a freshly initialised head attends to exactly one token out of 64, chosen by nothing but the random draw.

That is bad forward and worse backward, in a shape Chapter 5 already measured on a $\tanh$. A softmax committed to one entry has almost no derivative: the diagonal of its Jacobian is $w_i(1-w_i)$, zero at both ends. Over two thousand random rows:

At $d_k = 1024$, seven rows in ten are frozen before training starts, and a head that starts frozen cannot learn what to look at. Divided, the quantity is flat at 0.96 at every width and nothing saturates.

Now the part nobody publishes: does it change the final perplexity? Delete the division and train, at four head widths:

The first two rows come from the 3000-step budget above; the last three are a shorter run — 1500 steps, batch of 32, one head, no normalisation before the projections — with both variants under identical settings.

At $d_k = 32$ the division is worth nothing and the run without it is very slightly ahead. That is not a licence to drop it, because at 256 it is worth 18 points of perplexity and at 1024 it is worth 17. The mechanism is visible in the scores themselves:

The undivided head does not recover. It runs away: the standard deviation of its scores goes from 21 at initialisation to 5147, the attention entropy falls to zero, and 99.9 % of rows put more than 0.99 of their weight on a single token. Once a head is a hard selector its gradient is nearly zero and nothing pulls it back, so the collapse is stable. The divided head sits at a score standard deviation of 3.44 after the same training, which is a soft blend that can still be changed.

Vaswani et al. say exactly this and no more — they suspect the products "grow large in magnitude for large values of $d_k$" and divide.⁵ The word large is load-bearing, and the tables say where large starts: nothing at 32, everything by 256.

One head is one softmax row per position, so it holds one answer to "what is relevant here". Predicting the word after the in the animal that crossed the wet street needs the syntactic slot, the subject and the previous token at once, and one probability distribution cannot be concentrated in three places. So run several heads in parallel, each of width $d_{\text{model}}/h$, concatenate, and mix with one more matrix $W_o$: you have partitioned the width, not added to it.

Attention also does exactly one thing — it moves information between positions. Every operation in the code above is linear along the feature axis, and Chapter 5 proved what a stack of linear maps is. So each block also carries a small MLP applied to each position independently, expanding the width by four and coming back, with a GELU in the middle. The division of labour is worth memorising: attention mixes across positions, the feed-forward network computes within a position.

The full ladder, each row adding one piece to the row above it:

Learned weights beat uniform ones by 14 points of perplexity, which is this chapter's entire argument in one row. Four heads buy another 3 for 16,512 extra parameters. And the same head is worth 9 points more added than replacing: attention brings information in, it does not decide what a position is.

Now where the parameters actually sit, which surprises people who have only seen the diagram:

Two thirds of every transformer block is the feed-forward network, at every scale, because attention has four $d \times d$ matrices and the MLP has the equivalent of eight. Whatever a model knows, most of the parameters holding it are in the per-position MLP.

LayerNorm was built and measured in Chapter 6, and this chapter uses it as it was left there; residual connections were named and ablated there, and are built here. The "added, not replacing" rows above are residual connections, worth 188 points of perplexity for the average and 9 for one head. LayerNorm⁷ normalises each example across its features, and Chapter 6 gave the reasons it and not BatchNorm survived here — no dependence on the batch, no running statistics, identical in training and inference, indifferent to sequence length — every one of which becomes a requirement when you generate one token at a time for one user, which is where Chapter 13 ends up. It costs 768 parameters and buys 1.8 points of perplexity.

class Block(nn.Module):
    def forward(self, x):
        x = x + self.att(self.ln1(x))     
        x = x + self.ff(self.ln2(x))      
        return x

Look at where the normalisation sits: on the input of each sub-layer, with the residual path from input to output never normalised. That is pre-norm. The 2017 paper does the opposite, x = LayerNorm(x + Att(x)) — post-norm, which puts a LayerNorm on the residual path itself.

Xiong et al. explained the difference through the gradient at initialisation, which in a post-norm network is badly scaled with depth — the reason the original transformer needed a learning-rate warmup to train at all.⁸ Twelve blocks, 1000 steps, learning rate $3 \times 10^{-3}$:

gradient norm per block at initialisation, before any step
  pre-norm    block 1 0.0498 ... block 12 0.0657   ratio last/first  1.32
  post-norm   block 1 0.0977 ... block 12 0.1613   ratio last/first  1.65

  pre-norm,  no warmup          perplexity   37.82
  pre-norm,  200-step warmup    perplexity   37.62
  post-norm, no warmup          perplexity  308.05
  post-norm, 200-step warmup    perplexity   37.88

Post-norm without warmup is eight times worse, and post-norm with warmup matches pre-norm exactly. Warmup is not a general good practice here; it is a patch for a specific arrangement of the normalisation, and moving the LayerNorm removes the need for it. That is why essentially every model since 2019 is pre-norm, and why the 2017 diagram should be read as history rather than as a specification.

Delete the position embeddings and the model still trains; it simply cannot tell where anything is, and that is a symmetry rather than a training failure. Nothing in the attention score mentions $t$ or $i$ themselves, so permuting the input permutes the output: self-attention is permutation-equivariant. It is the average's order-blindness in a better disguise — the causal mask restores some order, since each position sees a different prefix, but within a prefix all orderings are alike.

Four ways to inject position, trained on 64-token windows and evaluated at 64, 128 and 256 — past any length they saw:

Learned absolute embeddings — one vector per position, added to the token — win at the trained length and then fall off a cliff, because position 100 was never in a batch and its embedding is still the random vector it started as. Sinusoids, the original choice, are computed rather than learned, from sines and cosines at geometrically spaced frequencies; the 2017 paper hoped that would extrapolate, and the table says it does not — the function is defined at position 200, but the model never learned to read it there. RoPE⁹ adds nothing and instead rotates query and key by an angle proportional to position, in two-dimensional slices; since rotating both sides of a dot product equally leaves it unchanged, the score ends up depending only on $t - i$, so position becomes relative for free and there is no table to run out of. It degrades, but it degrades. ALiBi¹⁰ is the simplest and the strangest result here: a linear penalty on the score proportional to distance, with a different slope per head. Its perplexity improves as the window grows past the training length, from 44.95 to 42.49, because the penalty is defined at any distance and every head keeps doing what it was trained to do.

The lesson outlasts the table: an architecture that cannot represent something is a different problem from one that never learned that range, and the second is the one that bites. It is also the machinery behind every "we extended the context to 128K" announcement — those are almost always re-scalings of a rotary encoding, and they are why Chapter 16 says the context limit moves rather than disappears.

Dropout is inherited the same way: it appears on the attention weights after the softmax, on each sub-layer's output before the residual addition, and on the embedding sum, doing exactly what Chapter 6 described. In large pretraining runs it is often set to zero, because a model that sees each token once is not in a position to overfit.

Two tensors in the layer have shape $n \times n$, where $n$ is the number of tokens: the scores and the weights after the softmax. Everything else — every projection, the whole MLP — is linear in $n$.

One attention layer, 512 wide, 8 heads, batch of one, float32, on a laptop GPU. Read the two millisecond columns for their ratios only: they are wall clock on an 8 GB laptop card that throttles from 1,785 MHz to under 300 MHz when it gets hot, so a cold run of this same code comes back seven to ten times faster and a busy one slower still. The megabyte columns are allocator byte counts and do not move.

  tokens   ms total    ms x4   ms projections   attn matrix MB    peak MB    MB x4
     128      2.246        -            1.324              0.5       14.6        -
     256      2.855     1.27            2.113              2.0       19.2     1.31
     512      5.761     2.02            3.105              8.0       34.4     1.79
    1024     16.414     2.85            4.008             32.0       89.1     2.59
    2048     51.573     3.14            9.989            128.0      296.1     3.32
    4096    225.432     4.37           20.176            512.0     1100.1     3.72
    8192    832.838     3.69           40.106           2048.0     4300.1     3.91
   16384   OUT OF MEMORY                                 8192.0

fitted exponent (log-log slope, last four rows):  time ~ n^1.91   memory ~ n^1.87

The x4 columns are the ratio to the row above, and a doubling of $n$ converges on exactly 4 for both time and memory — 3.91 at the last step against a theoretical 4. The projections column is the control: 4.0 ms at 1024 tokens to 40.1 ms at 8192, a factor of ten for a factor of eight. Linear, as advertised.

Then the last row. One attention layer, one sequence, no model around it, runs out of memory on an 8 GB GPU at 16,384 tokens — the score matrix alone would be 8 GB, being 8 heads times 16,384 times 16,384 times 4 bytes. Not the model; one intermediate tensor in one layer.

That is the physical fact underneath three later chapters. It is why a context window has a limit at all, which Chapter 16 turns into a price. It is why FlashAttention exists, computing the same result in tiles without ever storing the matrix — a memory optimisation before it is a speed one.¹¹ And it is the arithmetic behind the price of a long prompt, which Chapter 24 pays in an agent loop — a separate matter from that chapter's other finding, that a model also uses a long context worse, which it measures and declines to blame on this formula.

The 2017 paper describes an encoder-decoder: one stack reading the source with unmasked attention, a second generating the target causally, and a third kind of attention in the middle where the decoder's queries meet the encoder's keys. That is right for translation, where input and output are two sequences.

What won was the decoder-only half — one stack, causal throughout, input and output in the same sequence — and the reason is not elegance. "Predict the next token" runs on any text, so the training set is the internet rather than a parallel corpus, and everything becomes that one task: a translation is a document containing source then target, a question and its answer are a document, a conversation with a tool call in the middle is a document. Chapter 11 is about how that last one is manufactured. Encoders did not disappear — one sees the whole input at once, which is what you want when the job is to represent a text rather than continue it, and it is why Chapter 19's retrieval embeddings come from encoders and not from the model doing the chatting.

With the block defined, model size is arithmetic. Per block, with width $d$ and a four-times expansion: $4d^2 + 4d$ for $W_q, W_k, W_v, W_o$ with biases on all four, as GPT-2 has them — the table above leaves the bias off three of them, hence 2,304 fewer per block at $d = 768$; $8d^2 + 5d$ for the MLP; $4d$ for two LayerNorms — $12d^2 + 13d$, plus a token table of $V \times d$ and, for absolute positions, $n_{\text{ctx}} \times d$. For the shape of GPT-2 small — $d = 768$, 12 blocks, a vocabulary of 50,257, a context of 1024, the output layer sharing the embedding weights:

  token embeddings     50,257 x 768 = 38,597,376
  position embeddings   1,024 x 768 =    786,432
  one block                             7,087,872
  12 blocks                            85,054,464
  final LayerNorm         2 x 768 =        1,536
  total (weights tied)                124,439,808

Which is the published size of that model. The formula is not an approximation; it is the model. Note also that nearly a third of a small model is the embedding table, which is why vocabulary size is an architectural decision and not a preprocessing one — the trade-off Chapter 7 set up.

Perplexity is a number about a corpus. What one head does is a different question, and a model trained on a megabyte of Shakespeare is the wrong instrument for it: the honest thing to say about a 500,000-parameter model's attention map is that it is mostly not interpretable. So: a language where the question has a right answer.

The classic illustration is the animal did not cross the street because it was too tired, where it is the animal, against …because it was too wet, where one word moves the referent to the street. These are Winograd schemas¹⁴ — sentence pairs identical but for one word, where that word decides what a pronoun refers to.

They are also solvable by cheating, which is the part the tutorials skip. If the two candidates are an animal and a place, tired and wet identify the referent by category, and a model that only knows which words are present gets it right without knowing anything about order. Measured on that version of the task, with held-out animal/place pairs:

uniform causal average           held-out referent accuracy 100.0 %
one transformer block            held-out referent accuracy  91.7 %

The bag of words beats the transformer. Any demonstration built on that sentence proves nothing about attention.

So close the hole: draw both candidates from one pool of sixteen nouns, either of which can appear in either slot, and split the adjectives by role instead of category — four making it the crosser (tired, scared, slow, weak), four making it the crossed (wet, wide, busy, steep).

the {x} did not cross the {y} because it was too {adj} , so the {ref} waited .

Train as an ordinary next-token predictor, score one position — the word after so the — and build the held-out set from noun pairs whose reversed order was in training, so anything that knows which two nouns are present but not which came first must answer backwards.

Chance among the two nouns present is 50 %. The uniform average lands at 27.9 % and answers with the wrong noun of the pair exactly half the time — the signature of something that knows which words are there and nothing about their order, as the shuffle test predicted three sections ago.

Now the map: the attention at the position that has to name the referent, averaged over the four heads of each block, for the two sentences that differ by one word. A uniform average would put 0.067 on each of the fifteen visible tokens.

the animal did not cross the street because it was too tired , so the animal waited .
  blk 1  the:0.00 animal:0.70 did:0.00 not:0.00 cross:0.00 the:0.00 street:0.06
         because:0.00 it:0.00 was:0.00 too:0.00 tired:0.00 ,:0.05 so:0.00 the:0.19
  blk 2  the:0.00 animal:0.00 did:0.00 not:0.00 cross:0.00 the:0.00 street:0.00
         because:0.00 it:0.00 was:0.00 too:0.00 tired:1.00 ,:0.00 so:0.00 the:0.00

the animal did not cross the street because it was too wet , so the street waited .
  blk 1  the:0.00 animal:0.70 did:0.00 not:0.00 cross:0.00 the:0.00 street:0.06
         because:0.00 it:0.00 was:0.00 too:0.00   wet:0.00 ,:0.05 so:0.00 the:0.19
  blk 2  the:0.00 animal:0.00 did:0.00 not:0.00 cross:0.03 the:0.00 street:0.49
         because:0.00 it:0.00 was:0.00 too:0.20   wet:0.03 ,:0.00 so:0.00 the:0.25

Block 1 is identical in both sentences — 0.70 on the first noun, whatever the adjective is. That is not a failure but a proof: in the first layer the query at a position is a function of that position's own token and index, and the at position 14 is the same token in both sentences. A first-layer head cannot condition on a word it has not yet fetched. So block 1 does the only useful thing available to it and drags the first noun forward.

Block 2 is where the sentences part, and the same row across all eight adjectives shows the rule the model found:

For a crosser-adjective the second block spends its entire weight on the adjective, because the answer is already in the residual stream — block 1 put it there — and all it needs is confirmation. For a crossed-adjective it goes and fetches the other noun instead. That is a two-hop circuit: one head moves a candidate forward, a head in a later layer reads a token that decides whether to keep it. Composition across layers is the mechanism, and it is why one block reached 92.7 % and two reached 100 %.

It is also the shape of the best-documented circuit in real models. Induction heads — a previous-token head feeding a head in the next layer that completes the pattern [A][B] … [A] → [B] — are what Anthropic's interpretability work identifies behind a large part of in-context learning, and they form at an identifiable moment during pretraining. This chapter does not attempt that analysis: it is delegated, with both papers in the references, because reading circuits out of a real model is a research field and not a section.

Finally, the implementation. The thirty lines above, with their weights copied from PyTorch's own:

ours vs nn.MultiheadAttention           max |diff| = 1.7881393432617188e-07
ours vs F.scaled_dot_product_attention  max |diff| = 1.7881393432617188e-07

$1.8 \times 10^{-7}$ on outputs whose mean magnitude is 0.159: the same arithmetic in a different order, at float32 precision.

You have the architecture every model in the rest of this course is built from, and it is smaller than its reputation: a weighted average whose weights are learned, a per-position MLP holding two thirds of the parameters, two normalisations and two additions, stacked.

What you do not have is a model that knows anything, and stacking will not fix it by itself. Two blocks on this corpus reach a training perplexity of 14.49 and a validation perplexity of 40.57, against one block's 18.77 and 38.07 — more capacity, better on what it has seen, worse on what it has not, which is Chapter 6's table with a transformer in it. The distance between this model and the ones Chapters 14 to 30 talk to is not architectural. It is the same block, repeated more times, over vastly more text.

Which makes it an accounting problem, and the accounting is stranger than it looks. How much text, and where does anyone get it? How much arithmetic, and how do you estimate it before the money is spent? Given a fixed budget, is it better to make the model bigger or show it more data — and is there a correct answer, or only a fashion? Chapter 10 answers all three by measurement, and puts a price on the cheapest useful form of the question: what does it cost, today, to train a model like GPT-2 from nothing?

Three explanations of this material are better than this one at what they are for, and this chapter is written to be read alongside them. Jay Alammar's The Illustrated Transformer is the best picture of the data flow ever drawn. Harvard NLP's The Annotated Transformer is the 2017 paper with running code interleaved line by line. Andrej Karpathy's Let's build GPT: from scratch, in code, spelled out builds the same model live in two hours, and the ladder of ablations above is the same spine measured on a different corpus. For the interpretability question this chapter only touches, the primary sources are Elhage et al., A Mathematical Framework for Transformer Circuits (2021) and Olsson et al., In-context Learning and Induction Heads (2022), both from Anthropic's interpretability group.