Dot product: the multiply-and-add step at the heart of AI's math

What a dot product is

A dot product takes two lists of numbers of the same length, multiplies them position by position (first with first, second with second, and so on) and adds up the results. Two lists go in; one single number comes out.

For example, the lists (2, 3) and (4, 5) give 2×4 + 3×5 = 8 + 15 = 23. That is the whole operation. There is no hidden step.

What the video shows

The video walks through the move itself: pair up two lists of numbers, multiply each pair, add everything together. Its point is that this one move, repeated billions of times, is what every layer of a neural network computes. Hence the hook: the simplest math trick is doing most of AI's work.

An everyday example

Imagine a shopping receipt. You bought 3 apples, 2 loaves of bread and 1 carton of milk. The prices are 0.50, 2.00 and 1.20. Your total is 3×0.50 + 2×2.00 + 1×1.20 = 1.50 + 4.00 + 1.20 = 6.70.

You just computed a dot product: the list of quantities (3, 2, 1) against the list of prices (0.50, 2.00, 1.20). Any time you weigh several things by how much each one counts and add them up, such as a final grade built from exams and homework, you are doing the same thing.

Where it hides inside AI

• In a single decision. A perceptron, one of the simplest learning models, scores each example as the dot product of its weights (how much each measurement counts) with the measurements, plus one extra number called the bias. In our course, a factory part 18 mm wide and 47 g heavy is the list (18, 47). With made-up model weights of 2 and −1, the dot product is 2×18 + (−1)×47 = 36 − 47 = −11. Add the bias, and the sign of the result decides accept or reject.
• In a whole layer. A neural network layer is basically a stack of dot products, many of them computed side by side, one per output. Doing them all at once is what people call a matrix-vector product, where a matrix is simply a rectangle of numbers.
• In attention. Attention, the mechanism that lets a language model weigh which words to focus on, is covered in Chapter 9 of the course. At its core it is a dot product followed by a step that rescales the results.
• In similarity. In Chapter 19, measuring how alike two lists of numbers are (cosine similarity) needs only the dot product and the length of each list.

A common misconception

Many people assume you need a mountain of mathematics to understand AI. The course disagrees. It keeps its toolkit of linear algebra, the maths of lists and grids of numbers, deliberately short, and this is the heart of it:

• a vector is an ordered list of numbers;
• a matrix is a rectangle of numbers;
• the dot product combines two vectors into one number;
• a vector's length is the square root of its numbers squared and added up.

Advanced topics that fill a university linear algebra course, with names like determinants and eigenvalues, are left out on purpose. They matter elsewhere in mathematics, but this course never uses them, so it does not ask you to learn them.

Why it matters

When a neural network produces an answer, much of the arithmetic behind it is this one operation, repeated over and over at enormous scale. Knowing it will not make you an expert, but it removes a lot of the mystery. A great deal of what looks like machine intelligence is, underneath, multiplying pairs of numbers and adding them up.

Learn it step by step in Chapter 1 of our free course AI From Scratch: The Perceptron From Scratch: What a Neuron Computes.