Base model: the AI that continues your text instead of answering

What a base model is

A base model is what comes straight out of pretraining, the big first stage where an AI language model learns to guess the next piece of text across a huge library. It is a text completer: give it the beginning of something and it writes a likely continuation. It is not yet an assistant, the kind of AI that reads your request and tries to help you with it.

That difference sounds small. In practice, it changes everything about how the model behaves.

What the video shows

The video uses one classic example. Ask a freshly trained AI "What is the capital of France?" and, instead of answering, it may add more quiz questions. The model only knows how to continue text. Answering is something it has to learn in a second stage of training.

Why it writes more questions

A base model has exactly one skill: producing the text most likely to come next. So think about where a line like "What is the capital of France?" tends to appear in the enormous pile of text it learned from. Very often it sits in a list of exercises: a worksheet, a quiz, a set of revision questions. And in a list like that, the thing that usually follows one question is another question.

The model is not being stubborn or broken. It is doing precisely what it was trained to do.

An everyday example

Imagine someone who has read millions of pages but has never had a conversation. Hand them a card that says "Dear customer, we regret to inform you" and they will happily finish the letter. Hand them a card with a question copied from a worksheet and they will write the next question on the worksheet. They are fluent, but nobody ever taught them that a question can be a request aimed at them.

What a base model does not do

• It does not follow instructions. "Summarise this" is just more text to continue, not an order to carry out.
• It does not understand turn-taking, the back-and-forth of a conversation where you speak, it replies, and then it stops and waits for you.
• It has no sense of what to refuse. Give it the start of a harmful text and it will carry on writing it, because producing likely text is the only thing it was ever trained to do.

It also only knows what was in its training text, up to the date that text was collected, a limit known as the knowledge cutoff.

A common misconception

It is easy to assume that the chatbot you talk to and the model that came out of pretraining are the same thing. They are not. The helpful, conversational behaviour is added by a later stage of training built on top of the base model. The base model supplies the raw ability to write; the later stage teaches it to respond to a person.

Why it matters

Understanding base models explains why building an assistant takes more than one round of training, and why good behaviour cannot be taken for granted after pretraining alone. A base model has no idea what it should or should not say. It only knows what usually comes next.

Learn it step by step in Chapter 10 of our free course AI From Scratch: Pretraining an LLM: Data, Compute, Scaling Laws and Cost.