Reasoning tokens: the AI thinking you never see but still pay for

What reasoning tokens are

AI language models read and write text in small pieces called tokens, roughly a word or part of a word. Tokens are also the unit you pay in: providers bill by how many a request uses. A reasoning model, one that thinks a problem through before it replies, writes two kinds of tokens. Some are the answer you asked for. The others are its thinking, which you usually never get to read. Those hidden ones are reasoning tokens, and you pay for both kinds.

What the video shows

The video makes one point with two consequences. Reasoning AIs write out their thinking before answering. You often never see that thinking, but it still counts in two places: on the bill, and in the wait before the first visible word appears.

An everyday example

Imagine a mechanic who charges by the minute. You spend five minutes at the counter hearing what was wrong with your car, but the invoice also covers the forty minutes spent underneath it working the problem out. You never watched that work. It is still on the bill, and it is also the reason nobody said a word to you for a while.

How it is counted

This is where it gets practical, because providers do not all count the hidden thinking the same way:

• Most APIs (the connections that programs use to talk to a model) include the thinking in the output token count. So your bill includes it, and so does max_tokens, the limit you set on how long a reply may be. A limit that looks generous for the answer also has to leave room for thinking you will never read.
• Google's Gemini reports thinking tokens in their own separate field, outside the normal output count.

Those are two incompatible ways of describing the same thing. If you build anything that works out costs, or enforces a spending limit, across more than one provider, you first have to bring both styles into one common measure. Otherwise your numbers are not comparing like with like.

The silent pause

The second consequence surprises people the first time they meet it. The wait before the first visible word includes all of the model's thinking. In the course's example, a request shows nothing at all for eight seconds and then delivers its answer in about one. The connection has not frozen. The model is working.

For anyone building an app, that makes it a design problem rather than a network problem. A bare spinner with no explanation for eight seconds feels broken, even though nothing is wrong. The pause is normal; what needs fixing is how it is presented to the person waiting.

Why it matters

Reasoning tokens are easy to forget because you never read them. Yet they shape two things everyone notices about an AI feature: what it costs and how long it takes to reply. Knowing they exist explains a surprising bill and a puzzling silence. And choosing to spend more of that thinking on purpose is a whole idea of its own, known as test-time compute.

Learn it step by step in Chapter 12 of our free course AI From Scratch: Chain of Thought, RLVR and Test-Time Compute, Measured.