#215Dieses Video und der Text sind auf Englisch.
Stateless models: why your chatbot rereads the whole chat every time
An AI model keeps nothing between replies. Learn why every message resends the whole conversation, and why the context window is a limit, not a memory.
Im Detail
What a stateless model is
A stateless model keeps nothing from one request to the next. Chatting with an AI feels like talking to someone who remembers what you said five minutes ago. The model does not remember any of it. Every reply starts from a blank slate, and everything the model should know about your conversation has to be handed to it again, inside that same request.
What the video shows
The video makes one simple point. An AI model forgets everything after each reply. To keep a conversation going, the chat app sends the whole chat again every time, and the model rereads it from scratch. So the memory you feel comes from the app, which keeps the history and resends it, not from the model.
An everyday example
Imagine phoning a help desk where a different person picks up every time you speak, and none of them has ever heard of you. To get a sensible answer, you would have to give each new person the full written transcript of the call so far. They read it, reply, and hang up. The transcript is the only thing that carries on.
That is a chat with an AI. The transcript is the prompt: all the text sent to the model in one go, including the app's hidden instructions, every earlier message and your new question.
How it works
While the model writes an answer, it keeps working notes about every token it has read. A token is a small chunk of text, often a word or a piece of one. Engineers call these notes the KV cache. They are the model's entire working state while it writes, and they last for exactly one request.
When the answer is done, the computer that held those notes is free to work for someone else, and the notes are gone. On the model's side there is no session and no drawer labelled with your name. So the next request must bring everything with it, and the model rebuilds its notes by processing the entire prompt before producing anything new.
One consequence: anything an app wants the model to "remember" has to be written back into the text it sends.
A common misconception
People often call the context window "the model's memory". The context window is simply the most text a single request can hold, counting both the prompt and the answer. It limits how much the model can rebuild in one call. It is not a box that stores anything between calls. You are not filling up a memory; you are paying to set it up again, every single time.
Why it matters
Seeing the model as stateless explains a lot of everyday chatbot behaviour:
- A new chat knows nothing about an old one, unless the app copies something across into the prompt.
- A conversation longer than the window cannot be sent whole, so the app has to cut or shorten something.
- Every message pays to reread the history. That is why long chats get expensive, which is a story of its own.
It also changes how you build with these models. The history belongs to your app, and the model only ever sees what you send it right now.
Learn it step by step in Chapter 16 of our free course AI From Scratch: The Context Window, Tokens and the Bill, Measured.
Auch auf
Mehr aus diesem Kapitel
#215
Text mit AI-Unterstützung aus Kapitel 16 unseres kostenlosen Kurses geschrieben.