Post-training: how a text predictor becomes an assistant that answers

What post-training is

An AI language model is first trained on a vast amount of text, where it learns one skill: guess what comes next. That first phase is called pretraining. When it ends, the model is excellent at continuing text and poor at answering you. Post-training is a second phase of training that comes afterwards. It is a tiny fraction of the size of the first phase, and its job is to turn a machine that continues text into an assistant that replies to a request and then stops.

What the video shows

The video looks at a freshly trained AI, one that has only been through the first phase. Ask it something and it does not treat your words as a question to answer. It keeps writing whatever text would plausibly come next, so it rambles. Then a short second round of training teaches it two things: to answer, and to stop. The point of the video is that this step, not the size of the model, is what makes an assistant.

An everyday example

Think of the word suggestions on your phone keyboard. Tap the middle suggestion again and again and you get a sentence that sounds a bit like you, goes nowhere and never ends. A raw model is a far more capable version of that autocomplete. It is not broken. It is doing exactly the job it was trained for.

The course shows this with a real test. Asked to write a haiku about the sea, GPT-2, an older but properly trained model, replies "The sea is a beautiful place." and then writes that same line again and again, five times in total. Online, a request like that is often followed by ordinary prose about the sea, and text that was just written tends to repeat. So the model does the statistically sensible thing, which happens to be useless.

Now take Qwen2.5, a model built the same way with half a billion parameters (the adjustable numbers a model learns during training). That is about four times the size of this GPT-2, and still tiny by today's standards. After post-training, the same request gets a proper three-line haiku about the ocean.

How it works

Four times more parameters does not teach a model when to stop talking. The course is clear that the gap between the rambling and the haiku is not about size, not about the model's design and not about how much text it read. It comes from the second phase.

Post-training usually happens in two stages:

• Showing it examples. The model keeps training on requests paired with good replies, so it learns what an answer looks like and where it ends. This is called supervised fine-tuning.
• Teaching it preferences. People compare two replies and pick the better one, and the model is nudged toward the kind of reply people prefer. The classic recipe, called RLHF, trains a separate judge model on those choices and then lets the model practise against it. Later methods, called DPO and GRPO, cut out some of those moving parts.

A common misconception

It is tempting to think chat assistants became helpful simply by getting bigger. The haiku test says otherwise. A bigger raw model is still just an autocomplete. Taking turns, answering the question and stopping when done are behaviours the model is taught afterwards.

Why it matters

Every chat assistant you use went through this step. So when a raw model rambles or repeats itself, it is not being stupid: it simply never had this second training. And although post-training is small next to the first phase, it has an outsized effect on how useful a model feels.

Learn it step by step, starting with the haiku test, in Chapter 11 of our free course AI From Scratch: From Base Model to Assistant: SFT, RLHF, DPO and GRPO.