Test-time compute: making an AI better by letting it think longer

What test-time compute is

Compute simply means the amount of calculation a computer does. Test time is the moment a finished AI model is actually used to answer a question, as opposed to training time, when it is still being built. So test-time compute is the extra calculation a model spends while it answers: more thinking per question, instead of a bigger brain.

What the video shows

The video presents it as a third way to make AI smarter. Instead of building a bigger AI, you let it spend more effort on each question. And a small AI given time to work can sometimes beat a big one that answers instantly.

The two old dials, and the new one

The classic recipe for better AI had two main dials. Researchers describe them with scaling laws, rules of thumb for how a model improves as you make it bigger or train it on more data:

1. Model size: how many adjustable numbers, called parameters, the model has inside it.
2. Training data: how much text it learns from.

Both are paid for before the model is released, and neither can change once it starts answering questions. Test-time compute behaves differently:

| | Size and data | Test-time compute | |---|---|---| | When you pay | Once, before release | On every question | | Can you adjust it? | No, fixed after release | Yes, question by question | | What it costs you | A training budget | Waiting time and money per answer |

An everyday example

Imagine a tricky question lands on your desk at work. You could hire a more senior, more expensive expert for good, and pay that salary every single day. Or you could give the colleague you already have an extra hour on this one question. The first is a fixed cost whatever comes in. The second you pay only when a question deserves it. Test-time compute is the second option, applied to AI.

What the research found

A study by Snell and colleagues looked at what happens when you have a set amount of computing power to spend. For some problems, spending it while the model answers bought more accuracy than spending it on a larger model. That turns a common assumption upside down: the way to better answers is not always a bigger model.

Note the words "for some problems". This is not a claim that extra thinking always wins, or that small models are now better than large ones across the board.

Why it matters

This turns model choice into an engineering and product decision. A small model with room to think can beat a large model that replies straight away, and you decide the cost request by request instead of once per deployment.

The catch is time. More thinking means a longer wait and a bigger bill, because in a reasoning model that thinking is billed as reasoning tokens. Whether the trade is worth it depends on your product, not on the model. For example, someone asking for help with a hard maths problem may happily wait; a customer in a live chat may not.

Learn it step by step in Chapter 12 of our free course AI From Scratch: Chain of Thought, RLVR and Test-Time Compute, Measured.