#192הסרטון והטקסט שלו באנגלית.
Idempotency key: how to retry an AI request without paying twice
A timed-out request may still be running, so a blind retry can double the bill. See how one unique label per action lets the server answer only once.
בפירוט
What an idempotency key is
When an app sends a request and gets no answer in time, the natural move is to send it again. The catch: giving up on your side says nothing about what happened on the other side. Often the server did receive the first request and is still busy with it.
An idempotency key is a unique label your app creates once for each action, such as "answer this message", and attaches, unchanged, to every retry of that action. The long word idempotent simply means that doing something twice has the same effect as doing it once. The key is what gives a retry that property: the server sees a label it already knows, realises this is a repeat, and sends back the answer it already made instead of making a new one.
What the video shows
The video shows a request that times out while the AI is still working on it. The app retries, and the AI ends up answering, and billing, twice. Then it shows the fix: a unique tag on the action lets the server spot the repeat and answer only once.
An everyday example
Imagine phoning a pizza place and starting your order with a reference number you make up yourself, say 4721. The line drops before they confirm, and you call back. If you just order again, you may end up with two pizzas and two charges. If instead you say "this is order 4721 again", they check, see it is already in the oven, and do not make a second one. The number you chose at the start, and repeated on the second call, is the idempotency key.
What the numbers say
The course measured this with a test server that needs 780 milliseconds to produce an answer, and an app that gives up after 300 milliseconds and tries again.
| | What the app saw | Answers made and billed | |---|---|---| | No key | Two timeouts, no answer | 2 | | With a key | Timeout, then the stored answer, instantly | 1 |
Without the key, the server did the full job twice and would have charged twice, while the app received neither answer. With the key, the retry was not just harmless. It was the attempt that finally worked.
How to use it
- One key per action, not per attempt. Create it before the first try and reuse it on every retry. A fresh key on each try defeats the whole point.
- The server does the remembering. It stores the result under the key and replays it if the same key shows up again. Payment systems rely on the same trick, for the same reason.
- Know the limits. Not every AI provider accepts these keys when generating answers. If yours does not, and a request might already have gone through, the safe number of automatic retries is zero.
- Half-finished streams are a separate problem. If an answer arriving word by word breaks halfway, it usually cannot be replayed. You either start over and pay again, or keep the partial text and label it incomplete. That is a product choice, so make it deliberately.
Why it matters
Retrying is the standard response to network trouble, and the other ideas in this chapter, like backoff and rate limits, make retries smarter about when to happen. But none of them help if the retry repeats real, paid work. The key turns "maybe it ran, maybe it did not" into a question the server can answer for you.
Learn it step by step in Chapter 14 of our free course AI From Scratch: Your First Production LLM Call: Streaming, Retries, Timeouts.
גם ב־
עוד מהפרק הזה
#192
#193Streaming: why AI answers arrive word by word, and what it really buys
#194Finish reason: telling a finished AI answer from a cut-off one
#195Error triage: which AI errors deserve a retry and which never do
#196Exponential backoff: why a retry should wait twice as long each time
#197Thundering herd: when retrying apps move in lockstep and jam a server
הטקסט נכתב בעזרת AI, על בסיס פרק 14 בקורס החינמי שלנו.