Weight tying: why language models read and write with one table

What weight tying is

A language model works with tokens, the small pieces of text (whole words or bits of words) that it knows. For every token it keeps a list of numbers, called a vector, that describes it. It needs those descriptions twice: once to read the text coming in, and once to write, when it scores every token it knows to choose what comes next. Weight tying means using one single table of descriptions for both jobs, instead of keeping two separate ones.

What the video shows

The video starts from the usual picture: a language AI keeps a huge table for reading text pieces, and it needs a second one for writing them. Then it shows the trick: the same table does both jobs. The payoff it points to is double: fewer settings to learn, and better predictions.

Its opening line calls this the trick almost every language AI uses. The course is more careful: it says the trick is close to universal in small language models, and the numbers below show why.

An everyday example

Imagine learning a new language with two flashcard decks. One deck helps you recognise a word when you hear it. The other helps you pick the right word when you speak. Every time you learn something new about "harbour", you have to update both decks, and they can slowly drift apart. Now imagine throwing one deck away and using the same card for listening and for speaking. You carry half the cards, and every card gets used in both directions.

That is the spirit of weight tying: a token keeps one identity, whether the model is reading it or writing it.

How it works

Reading is a lookup. Each token has an ID number, and the model fetches that token's row from a table, often called the embedding table.

Writing looks different but needs the same ingredient. At the end of its work, the model holds one vector that sums up "what I want to say next". To turn that into a choice, it gives every token in its vocabulary a match score: a multiply-and-add comparison between its vector and one vector per token. So the output layer (the final step, which produces those scores) also holds one vector for every token.

That makes two tables with one row per token and the same shape. Tying says: keep just one. Use it forwards to look up rows on the way in, and turned on its side to score every token on the way out.

Why the table is worth sharing

These tables are big. In BERT-base, a well-known language model, the vocabulary has 30,522 tokens and each one gets 768 numbers. That comes to about 23.4 million numbers, or 21.4% of the model's roughly 109 million parameters (the adjustable numbers a model learns during training). In smaller language models the table takes an even larger share, so sharing it pays off most there. A separate writing table of the same shape would cost as much again.

Why it matters

Saving memory usually sounds like a compromise. Here it is not. The course reports that tying cuts the number of parameters and improves perplexity (a measure of how surprised a model is by real text, where lower is better) at the same time. Getting smaller and better in one move is rare enough to be worth remembering. It is also a neat lesson in design: when two jobs need the same knowledge, one shared copy can beat two separate ones.

Learn it step by step in Chapter 8 of our free course AI From Scratch: Next-Token Prediction: Embeddings, and What Perplexity Means.