Think step by step: why a famous prompt works, and when it backfires

What "think step by step" is

"Let's think step by step" is a short sentence people add to the end of a question for an AI. In 2022, researchers (Kojima and colleagues) found that this one sentence made language models noticeably better at problems with several steps, with no worked examples needed. It is a classic example of prompting, the craft of wording your requests to an AI.

What the video shows

The video tells the story of a magic phrase that stopped being magic. In 2022, adding "let's think step by step" made AI answers better. But the phrase taught the model nothing new. Today's reasoning AIs, which are trained to work through problems on their own, already do it, and adding the phrase can get in their way.

Why the phrase worked

It is tempting to say the model "started thinking". The real explanation is plainer: it computed for longer.

• There is no "try harder" dial. The kind of AI behind today's chatbots does the same fixed amount of work for every token it produces (a token is a word or part of a word), whether the question is easy or hard.
• Each written token buys another round of work. In the course's test, a small model answered 24 two-step word problems. Told to give only the number, it produced about 2 tokens and got 0 right. Told to think step by step, it wrote about 145 tokens and got all 24, with the same model and the same settings.
• The words become notes it can read. Once the model writes "5 × 13 = 65", that result sits in its input for the next step, instead of having to be carried along silently.

That last point explains a curious detail: asking a model to "think silently, then answer" does not help. The silent thought has nowhere to live.

An everyday example

Imagine asking a friend two questions. First: "What is the capital of France?" Either they know it or they do not, and saying "take it step by step" changes nothing. Second: "A room has 3 chairs and 2 dogs. How many legs?" Here, writing "3 × 4 = 12", then "2 × 4 = 8", then "12 + 8 = 20" really helps, because the last step depends on the first two.

That is the pattern the course describes. The phrase helps most when step two needs the result of step one, and does nothing for a single fact lookup.

When it backfires

Newer reasoning models have been trained to produce long working by themselves. For them the phrase is at best redundant. At worst it can hurt, because it may nudge the model to swap its own longer reasoning for a shorter version shaped by your prompt. Think of handing an experienced pilot a five-line checklist when they normally run a longer one of their own.

Why it matters

The phrase is a good lesson in how to read AI tricks. It never unlocked a hidden power. It gave the model more room to compute and a place to keep its notes. Knowing that tells you when to use it (multi-step problems, on models that do not already reason) and when to leave it out (simple lookups, and models trained to reason on their own).

Learn it step by step in Chapter 12 of our free course AI From Scratch: Chain of Thought, RLVR and Test-Time Compute, Measured.