Sari la conținut

Acest videoclip și textul său sunt în engleză.

Prompt order: the same words, rearranged, can double AI accuracy

AI Concepts #205

Rearranging the same three blocks of a prompt took a small AI from 27% to 55% accuracy. See which order won, and the simple layout rule that follows.

În detaliu

What prompt order is

Most prompts are built from a few blocks: the rules the AI must follow, the data it should work on, and the instruction that says what to do with it. Prompt order is simply the sequence in which you stack those blocks. It sounds like a matter of style. It is not: the very same words in a different order can change how often a model gets the answer right.

What the video shows

The video rearranges the same three blocks of a prompt and shows a small AI's accuracy moving from 27% to 55%. Its takeaway: put the task first, the data last, and leave nothing important in the middle.

The numbers come from the course. A small model sorts customer support tickets into queues. The prompt has three blocks: the rules describing each queue, the ticket itself, and the instruction. The course tried all six possible orders, with exactly the same characters every time, on 60 tickets each:

| Order of the blocks | Correct | |---|---| | rules, instruction, ticket | 55.0% | | rules, ticket, instruction | 50.0% | | ticket, rules, instruction | 36.7% | | instruction, ticket, rules | 35.0% | | instruction, rules, ticket | 28.3% | | ticket, instruction, rules | 26.7% |

The best order is roughly twice as accurate as the worst. The margins of error around those two scores do not overlap, and a ticket-by-ticket check agrees: going from the worst order to the best, 21 tickets flipped from wrong to right and only 4 went the other way. This is not luck.

An everyday example

Imagine forwarding a colleague a long email chain with your real question tucked into the middle, between some background and a pasted report. There is a good chance they reply about the report and never answer the question. Put the question in the opening line, paste the report at the bottom, and you get an answer to what you actually asked.

How to lay out a prompt

Look at the pattern, not just the winner. The two best orders both open with the rules, and the winner saves the ticket, the data, for the very end. The worst does the opposite: it opens with the ticket and pushes the rules to the end. The course links this to an effect researchers (Liu and colleagues) called lost in the middle: models make more reliable use of what sits at the start and the end of a prompt than of what sits in the centre. The table is not perfectly tidy, though. Another order that also ends with the ticket (instruction, rules, ticket) came second from the bottom, so treat the rule below as a starting point to check, not a law.

The practical rule that falls out:

  • The task and its rules go first.
  • The data goes last.
  • Nothing you care about gets buried in between.

This is about the order of the prompt's own blocks. It is different from the order of answer options inside a list, and from which chat message the words go into; both of those are separate levers.

Why it matters

Reordering costs nothing: no new words, no bigger model, no extra fees. In this test it was worth about 28 percentage points. The honest caveat is that these numbers come from a small open model with about half a billion parameters (the adjustable internal values a model learns), and the course expects a bigger model to be more robust to most of these effects. The direction still matches published research, and checking your own prompt takes minutes: try a few orders on the same test cases and keep the one that wins.

Learn it step by step in Chapter 15 of our free course AI From Scratch: Prompt Engineering, Measured: What Changes the Output.

Și pe

Mai multe din acest capitol

Text scris cu asistență AI din capitolul 15 al cursului nostru gratuit.