Knowledge cutoff: why an AI can be confidently wrong about recent news

What a knowledge cutoff is

An AI chatbot learns from a huge pile of text gathered up to a certain day. That day is its knowledge cutoff. About anything that happened afterwards, the model does not know a little, or know vaguely. It knows nothing at all.

Every model has a cutoff. Meta's Llama 3.1, for example, has one of December 2023, stated in its model card (the fact sheet published alongside the model).

What the video shows

The video explains that every AI stopped "reading" on a certain date. After that date it knows nothing, yet it may still answer confidently, which is how it ends up getting recent news wrong. It also says that looking up fresh information fixes this. Strictly, that is a workaround rather than a fix, for reasons explained below.

An everyday example

Imagine a friend coming home after a year at a remote research station with no internet. Ask them who won last month's big match and a sensible friend says, "No idea, I've been away."

Now imagine a friend who never learned to say that. They answer at once, calmly and convincingly, with whatever sounds most likely based on what they knew a year ago. They are not trying to fool you. Admitting they have been away simply is not something they do.

That second friend is how a language model behaves past its cutoff.

Why it does not just say "I don't know"

A language model is trained to produce text that sounds right, not to keep track of which dates it has and has not seen. Saying "that is after my training data" was never a behaviour its training taught it. So instead of stopping, it fills the gap with smooth, plausible sentences. The word for this is confabulating: making something up fluently, without any sign that it is made up.

That is what makes the cutoff tricky. A wrong answer about last week reads exactly like a right answer about last year.

It belongs to the data, not the app

A common assumption is that a newer app, or a model released more recently, must know more recent things. Not necessarily. The cutoff is a property of the training data, not of when or where the model is being used. A model used two years after its data was collected still knows only what was in that data on the day collection stopped. Wrapping it in a fresh app does not move the date.

How to live with it

• Check the date. For anything recent, such as news, prices, rules or results, assume the model may be out of date.
• Look for sources. Tools that first search the web or a set of documents, then answer using what they found, work around the cutoff. This is called retrieval, and the course covers it in Chapter 19.
• Remember what retrieval does and does not do. The model itself has not learned anything new. It has been handed fresh text to read at the moment you ask. Take that text away and the cutoff is back.
• Treat confident detail about recent events with suspicion. A smooth answer is not evidence that the model knows.

Why it matters

Knowing about the cutoff turns a confusing failure into a predictable one. The model is not broken when it gets last week wrong; it was never shown last week. Once you know roughly where the date is, you know which questions need a fresh source.

Learn it step by step in Chapter 10 of our free course AI From Scratch: Pretraining an LLM: Data, Compute, Scaling Laws and Cost.