Query, key, value: the three jobs every word does inside attention

What query, key and value are

Attention is the part of a modern language model where each word gathers information from the other words. To make that work, every word is turned into three versions of itself. Each is a learned list of numbers with its own job:

• Query: what this word is looking for.
• Key: what this word offers, so that other words can find it.
• Value: what this word actually hands over when it is picked.

A word compares its query with the key of every word it is allowed to see. Good matches get more weight. Then the word collects a blend of all the values, mixed according to those weights.

What the video shows

The video walks through those three roles. Inside attention, each word asks what it is looking for, advertises what it offers, and shares its content with the words that match well. What comes out is a blend of all the words, not a lookup of one.

An everyday example

Imagine walking into a busy café with a question in your head: who here knows about bikes? That question is your query. Everyone wears a badge saying what they know about: that is their key. You do not pick one person. You hear everyone at once, but each voice is turned up or down depending on how well their badge matches your question. What reaches you is a mix of what people actually say, their value, mostly from the best matches.

The badge and the talk are different things: a badge says "cycling", the talk is last weekend's route. That is why keys and values are kept apart.

Why one role is not enough

First, the weights cannot be fixed numbers, because who matters depends on the sentence. In an example like "the animal was tired, so it lay down", the word "it" should look at "animal". A fixed rule such as "always look four words back" can never know that.

The simplest way to compare two words is to multiply their lists of numbers item by item and add it all up, a similarity score called the dot product. Doing that with the raw words fails in two ways:

• Every word would mostly pay attention to itself, since its numbers line up perfectly with themselves.
• Matches would go both ways. If "it" focuses strongly on "animal", then "animal" would focus strongly on "it" too. Language is lopsided: an adjective leans on its noun much more than the other way round.

Giving each word a separate query and key fixes both, because a word can search for one thing while advertising another. The separate value fixes one more problem: the features that make a word easy to find are not the same as the features worth passing on.

A common misconception: it is not a database

The course warns against the popular database analogy. A database returns one row that matches exactly. Attention returns a weighted blend of every word, and the blend is the point: the result at "it" is not simply "animal", it is a mix that is mostly "animal", and later layers use that "mostly". Nothing is stored and nothing is looked up. Three grids of numbers are multiplied and a softmax is taken (the step that turns scores into shares adding up to 100%).

Real document search, covered later in the course, is a different thing.

Why it matters

Query, key and value sit at the core of every transformer. In the course's code, a whole attention head (one complete copy of this mechanism) is about thirty lines, and most of them only arrange shapes. Score, hide the future, turn scores into weights, mix the values: that is attention.

Learn it step by step in Chapter 9 of our free course AI From Scratch: Attention and the Transformer Block, Derived From an Average.