Less is more (LIMA): how 1,000 good examples shaped an AI assistant

What LIMA is

LIMA, short for "Less Is More for Alignment", is a study about a practical question: how much teaching does a well-read AI need before it becomes a useful assistant? Its answer was a small number: about a thousand carefully chosen examples were enough. Each example, called a demonstration, is simply a question paired with a well-written answer.

To see why that number is small, remember that building a chatbot happens in two big stages. In pretraining, the model reads a huge amount of text. In post-training, that well-read model is shaped into an assistant that answers you helpfully. LIMA is about the second stage, and how little data it might need.

What the video shows

The hook is the number itself: 1,000 examples, and that was all it took. The point behind it is that an AI which has already read a huge amount knows most of what it needs. A study found that roughly 1,000 well-chosen examples were enough to turn it into a solid assistant.

An everyday example

Imagine hiring an experienced journalist to write your company newsletter. You do not teach them grammar, research or how to tell a story. They arrived with all of that. What they need is to see how you do things here: the tone, the length, the layout. A dozen of your best past issues will teach them more than a pile of rushed ones, because every example they study is one they will copy.

That is the LIMA idea in one scene. The knowledge came with the hire. The examples only point at which of their existing skills you want, and in what shape.

How it works

The explanation the researchers proposed has three parts:

• Pretraining did the heavy lifting. By reading so much, the model already picked up its knowledge, and even what a good answer tends to look like.
• Post-training mostly selects. It is not where the knowledge comes from. It chooses which behaviours the model already has should come to the front.
• So quality beats quantity. If each example is steering rather than teaching, every single one counts. The quality of post-training data then matters much more than how much of it you have.

Notice the careful wording. This is a proposed explanation, not a proven law. What can be said is that the field has acted since as if it believes it.

A common misconception

LIMA does not mean a thousand examples can build an AI from nothing. They only work because the model had already read a vast amount first. Take away that reading and there is nothing for the examples to steer. The lesson is narrower and more useful: once a model is well read, the final shaping step depends more on choosing excellent examples than on collecting many.

Why it matters

It changes where the effort should go. If post-training mostly brings out behaviour that is already there, the people building assistants gain more from writing and checking a small set of excellent examples than from piling up large volumes of average ones.

If the explanation holds, it also tells you where an assistant's knowledge really comes from: the long reading stage, not the final polish. The polish mostly decides how that knowledge is presented to you.

Learn it step by step in Chapter 11 of our free course AI From Scratch: From Base Model to Assistant: SFT, RLHF, DPO and GRPO.