Verifiable rewards: training AI on answers a program can check

What verifiable rewards are

To train an AI model with rewards, something has to tell it whether each answer was good. For most questions that judgement is fuzzy. "Was this a helpful reply?" has no answer key, so earlier training relied on people rating answers, and on a second AI model trained to imitate those ratings, to play the judge.

For some tasks, though, a program can check the answer directly:

• a maths answer either matches the correct value or it does not;
• a piece of code either passes its tests or it does not;
• a proof either checks out or it does not.

In those areas the learned judge can be swapped for a verifier, a program that simply says right or wrong. Training a model this way is called reinforcement learning from verifiable rewards, or RLVR. Reinforcement learning just means learning by trial and error, guided by rewards.

What the video shows

The video explains that in maths and code, a program can tell whether an answer is right. That check replaces human judges during training, and long, flattering answers cannot fool it.

An everyday example

Imagine two ways of grading homework. An essay goes to a tired teacher, who might be swayed by a long, confident answer full of nice phrases. A maths worksheet is marked against the answer key: 42 is right or it is not, however polished the working looks. Verifiable rewards give AI training that answer key.

How it works

A training recipe built for exactly this setting is called GRPO. In plain terms:

1. Give the model one problem and let it attempt it several times.
2. Check every attempt with the program.
3. Score each attempt against the average of the group rather than on its own, so the model learns which of its attempts beat its own average on that problem.

No human raters are needed, and there is no separate judge model to build or train.

Why a program makes a better judge

A learned judge can be gamed. A classic example is length: if the judge tends to favour longer answers, the model learns to pad its answers out. A test that runs the code does not care how long or polite the answer is. You cannot flatter a unit test, the small automatic check programmers write to confirm that code does what it should.

This is also why the idea was such a turning point. Asking a model to reason step by step happens at the moment you use it. Verifiable rewards let the same behaviour be trained into the model, and that shift is what produced today's generation of reasoning models.

The limits

It only works where a program can decide what "correct" means. A good email, a kind reply or a fair summary have no answer key. And checking only the final answer has a blind spot of its own: a right number reached through wrong steps earns exactly the same reward.

Why it matters

Verifiable rewards swap an expensive judge that can be flattered for a cheap one that cannot. That is a big deal wherever a clear right answer exists, and a reminder that the trick does not stretch to questions without one.

Learn it step by step in Chapter 12 of our free course AI From Scratch: Chain of Thought, RLVR and Test-Time Compute, Measured.