Chain of thought: why letting an AI show its work changes the answer

What chain of thought is

Chain of thought means letting an AI model write out its in-between steps before it gives a final answer. Instead of jumping straight to a number, it writes something like "first this, then that, so the answer is 45". Nothing inside the model changes. Same model, same knowledge. The only difference is that it gets to work on the page before it commits to an answer.

What the video shows

A small AI model is given 24 short word problems. When it is told to reply with just the number, it gets 0 of them right. When it is allowed to write its steps first, it gets all 24. Same model, 0% versus 100%. The explanation it gives is short: writing gives the model more effort and a notepad.

An everyday example

Imagine splitting a restaurant bill with three friends. Someone asks for your share the second the bill lands. You have to add the drinks, add the food, take off a voucher and divide by four, all in one go, in your head. You will often be off. Now grab a napkin and jot down each subtotal. Every step is easy, and you can read your last line before doing the next one. The napkin did not make you smarter. It gave you room to work and a place to keep the numbers.

How it works

Two things happen when the model writes its steps.

• More effort. A transformer, the kind of AI behind today's chatbots, does a fixed amount of work for every token it produces (a token is a word or a piece of a word). An instant answer gets one round of that work. Every extra token of written working is another full round.
• A notepad. Whatever the model writes becomes part of what it reads next. A half-way result stops being something it has to hold in its head and becomes text on the page.

Where the idea came from

The technique took off in 2022, in two steps. First, Wei and colleagues showed that adding a few worked examples to the prompt, where each answer comes after its reasoning, made models much better at arithmetic and common-sense questions. Then Kojima and colleagues found you could drop the examples: one added sentence, "Let's think step by step", recovered much of the same gain.

A common misconception

It is tempting to think chain of thought teaches the model to reason. The course argues the opposite. The huge pile of text a model first learns from, called its pretraining data, is full of worked solutions. The ability to lay out steps was already there. The prompt only points the model toward that habit. If a single sentence is enough to switch a behaviour on, the model must have picked it up long before.

Why it matters

Chain of thought is a simple, practical lever, though not a free one: all that written working costs extra tokens. For problems where each step builds on the last, asking for the working can be the difference between failure and success. Knowing why it works also tells you its limits, and it explains why newer "reasoning" models are trained to do this on their own instead of waiting to be asked.

Learn it step by step in Chapter 12 of our free course AI From Scratch: Chain of Thought, RLVR and Test-Time Compute, Measured.