King − man + woman = queen? The hidden rule behind AI's famous demo

What the famous demo claims

Word vectors are lists of numbers that stand for words, learned so that words used in similar ways get similar lists (word2vec is one well-known way to make them). The famous demo does arithmetic with them: take the vector for "king", subtract "man", add "woman", then look for the word whose vector is closest to the result. The answer, we are told, is "queen". It is often presented as proof that the model has grasped gender and royalty.

What the video shows

"King − man + woman = queen" is the classic demo. But with nothing filtered out, the word closest to the result is usually "king" itself. The demos quietly remove the input words before picking an answer. The effect is real, just much weaker than it looks.

What the numbers say

The course runs the demo on vectors it trained on about 17 million words of Wikipedia:

• Nothing excluded: the closest word to king − man + woman is "king" (0.693), followed by "elizabeth", "wife" and "woman".
• Input words excluded: "elizabeth", "wife" and "mary" come first. "Queen" only makes fourth place.

This is not one unlucky example. A standard test set asks thousands of questions of the form "a is to b as c is to what?", such as "Paris is to France as Rome is to what?". The course's vocabulary can answer 4,103 of the meaning-based ones. Without the exclusion rule, the winning word is one of the three inputs 99.8% of the time.

The hidden rule, and how much work it does

The standard way of scoring deletes the three input words before looking for the answer. That rule is legitimate, since the answer to an analogy should not be one of its own inputs. But it is doing much of the heavy lifting:

| How the answer is picked | Meaning questions right | |---|---| | Arithmetic, inputs excluded (standard) | 17.0% | | Arithmetic, nothing excluded | 0.1% | | No arithmetic: word nearest to "c", inputs excluded | 13.1% |

Look at the last row. Skip the subtraction and addition entirely, simply return whatever sits closest to the third word, and you still keep 77% of the standard score.

An everyday example

Imagine a multiple-choice test where the teacher quietly crosses out the three most tempting wrong answers before the student picks. The score goes up, but not because the student understood more. That is roughly what happens here: much of what looks like reasoning by analogy is plain closeness, plus a rule that forbids the obvious answers. A researcher named Linzen found the same pattern on properly trained vectors.

A common misconception

The demo is often read as "the model understands gender". The honest version is more modest. The arithmetic does carry a real signal, since it beats the no-arithmetic shortcut (17.0% against 13.1%). But the gap is small. These vectors are also small, so read the percentages as a shape rather than a record; the course notes that the same shape holds at every scale.

Why it matters

When a single striking example is passed around as proof, ask two questions: how was the answer scored, and what does a simple shortcut get? Here, the famous result leans on a quiet rule, and most of its score comes from nearness alone.

Learn it step by step in Chapter 8 of our free course AI From Scratch: Next-Token Prediction: Embeddings, and What Perplexity Means.