#203Dieses Video und der Text sind auf Englisch.
Few-shot prompting: how many examples an AI prompt really needs
Two worked examples lifted a small AI's accuracy seven points; six more added nothing. Learn how many examples to use, which ones, and why order matters.
Im Detail
What few-shot prompting is
Few-shot prompting means putting a few solved examples into the prompt before the real question: here is a request and its right answer, here is another. The model picks up the pattern and applies it to the new case, with no retraining at all. That ability to learn from examples inside the prompt, called in-context learning, is what made GPT-3 famous. The useful question today is not whether it works, but how many examples are worth paying for.
What the video shows
Showing a small AI two solved examples raised its accuracy by about seven points. Adding more did not help much, and which examples you pick, and the order they come in, mattered just as much as how many there were.
The numbers come from the course. A small model sorts customer support tickets into queues. The examples go in as earlier turns of the chat: a user message with a ticket, then an assistant message with the right queue. Each number of examples was tried five times, with a different random pick from a pool of sixteen labelled tickets each time:
| Examples | Average accuracy | Gap between the best and worst pick | |---|---|---| | 0 | 76.7% | n/a (nothing to pick) | | 1 | 78.7% | 1.7 points | | 2 | 83.7% | 6.7 points | | 4 | 81.7% | 8.3 points | | 8 | 83.7% | 10.0 points | | 16 | 89.3% | 8.3 points |
The first two examples were worth about seven points. The six after them added nothing you can measure. Going all the way to sixteen added roughly five and a half more. The gains came in jumps: up, then flat, then up again.
An everyday example
Imagine teaching a temp to sort the office mail by first showing them a few letters you have already sorted. Two good examples, say a bill and a complaint, get them going quickly. But if the four examples you grab all happen to be bills, they learn "everything goes to accounts", not how to sort. More examples of the same kind do not fix that.
How to use examples well
The biggest surprise is in the last column. With eight examples, simply which eight you happened to draw moved accuracy by 10 points, bigger than the whole gain from two examples to eight, which was zero. With sixteen, the pool was used up, so every run had exactly the same sixteen examples, just shuffled. Order alone moved accuracy by 8.3 points. Other researchers (Lu and colleagues) found the same: the order of examples matters about as much as their number.
The course turns that into four habits:
- Start at zero and add examples only when a measurement shows they help. The first two are usually worth it. After that you are guessing, and every example costs tokens (the chunks of text a model reads, which you pay for) on every single call from then on.
- Treat the choice as part of the prompt. A couple of well-chosen examples beat a larger batch copied from the top of a spreadsheet.
- Try a few orders once, then freeze the best one. It is free and the effect is real.
- Balance the answers. Examples that all share one label teach that label, not the task.
Why it matters
"Add more examples" is one of the most common prompt reflexes. Measured, the payoff is lumpy, and the choice and order of the examples can move results as much as their count. The numbers come from a small open model with about half a billion parameters, so the exact figures will differ on bigger models, but the habit transfers: measure before adding, and treat your examples as something to tune.
Learn it step by step in Chapter 15 of our free course AI From Scratch: Prompt Engineering, Measured: What Changes the Output.
Auch auf
Mehr aus diesem Kapitel
#203
#204Option order bias: when an AI picks a position, not an answer
#205Prompt order: the same words, rearranged, can double AI accuracy
#206System prompt: why where you put AI instructions changes results
#207Prompt formatting: how quotes and tags quietly shift AI accuracy
#209Prompt myths: what tips, pep talks and threats did in a real test
Text mit AI-Unterstützung aus Kapitel 15 unseres kostenlosen Kurses geschrieben.