Multi-Agent Orchestration: Five Patterns, and When One Wins

A Portuguese company writes in about invoice FT-2026-0918. The email says the VAT looks wrong, and attaches the invoice: net EUR 248.00, VAT charged at 21 %, EUR 52.08, total EUR 300.08.

The facts needed to answer live in three places, and only one of them is in the email:

Put the three together and the invoice is wrong: reverse charge applied, the VAT should have been zero, a credit note for EUR 52.08 is owed. Look only at the invoice and it is arithmetically perfect — 248.00 plus 52.08 is 300.08 — and you will say so.

The email does state "we are a Portuguese company". That is a claim, not a record, and no billing system issues a credit note on a claim. The trap is not a trick: it is the ordinary shape of business work, where the decision needs a fact nobody thought to fetch.

Everything above runs against a scripted provider in the style of Chapter 23's, with exactly one rule:

An answer may only use a fact that is in its prompt.

The "model" asks for each tool it has, once, in catalogue order, then applies a fixed rule to the text it can see. Nothing is scripted per arrangement, so the differences in that opening table are not claims about model intelligence: they are information routing, measured. A real model adds its own failures on top; it does not remove these.

The five names below are Anthropic's, from Building effective agents, which is where this vocabulary settled.¹ None of the five ideas is new, and saying which house named what — and which idea is older — is half the value of knowing them.

/* 1. Prompt chaining: a fixed pipeline. The control flow is yours. */
export async function chain(steps: Step[], first: string) {
  let carry = first, all = first;
  for (const s of steps) {
    const r = await step(s.role, s.system, s.accumulate ? all : carry);   
    carry = r.text;
    all = `${all}\n${r.text}`;
  }
  return carry;
}

/* 2. Routing: one cheap call picks the branch. The fallback is not a model. */
export async function route<T>(input: string, classify: Classifier,
                               routes: Record<string, Branch<T>>, fallback: Branch<T>) {
  let label: string | undefined;
  try { label = await classify(input); } catch { label = undefined; }
  return ((label && routes[label]) || fallback)(input);                    
}

/* 3. Parallelisation. The pattern IS this line. */
export const parallel = <T>(workers: Branch<T>[], input: string) =>
  Promise.all(workers.map((w) => w(input)));                              

/* 4. Orchestrator-workers: an agent behind a tool. Chapter 18's interface, unchanged. */
export function agentTool(o: WorkerSpec): Tool {
  return {
    name: o.name, description: o.description, readOnly: true,
    parameters: { type: "object", properties: { question: { type: "string" } } },
    async run(args: { question: string }) {
      const child = newRun(o.system, args.question);          // its own window
      await runTracked(child, o.tools, o.usage);              // its own limits
      const conclusion = child.output ?? "no result";
      if (!o.carryFindings) return conclusion;                             
      return `${conclusion}\nFINDINGS ${evidence(child)}`;                 
    },
  };
}

/* 5. Evaluator-optimiser: make, judge, remake. Rounds are calls. */
export async function refine(make: Make, judge: Judge, maxRounds: number) {
  let draft = "", feedback: string | undefined;
  for (let r = 1; r <= maxRounds; r++) {
    draft = (await make(feedback)).text;
    const j = await judge(draft);
    if (j.ok) return { draft, rounds: r };
    feedback = j.note;
  }
  return { draft, rounds: maxRounds };
}

That is the whole toolkit: five functions, no framework, and the parallel one is a single line — which is the point of writing it out rather than drawing it. Now each in turn, with its ancestry, its price, and the case where it is wrong.

Prompt chaining "decomposes a task into a sequence of steps, where each LLM call processes the output of the previous one".¹ The idea predates language models: it is a pipeline, with the pipeline's trade — clarity in exchange for a control flow fixed before the data arrives.

Four steps for our task: extract the invoice fields, check the arithmetic, decide what is owed, write the reply. Here it fails in two different ways, which teaches more than failing once.

--- relay: each step sees only the previous step's output
extract: FIELDS invoice_id=FT-2026-0918 net=248.00 vat_rate_applied=21 vat_amount=52.08 ...
check:   ARITHMETIC ok 248.00+52.08=300.08
decide:  VERDICT=unknown reason=no_invoice_in_context
draft:   "we are looking into invoice FT-2026-0918 and will come back to you."

--- accumulating: each step sees the email and everything produced so far
extract: FIELDS invoice_id=FT-2026-0918 net=248.00 vat_rate_applied=21 ...
check:   ARITHMETIC ok 248.00+52.08=300.08
decide:  VERDICT=invoice_correct reason=net_248.00_plus_21pct_vat_52.08_equals_300.08
draft:   "we have checked FT-2026-0918 and it is correct... Nothing is owed back."

The relay chain cost $0.001940 and lost the invoice fields between steps two and three, because step three was handed a sentence about arithmetic and nothing else. It produced a holding message: useless, and visibly useless.

The accumulating chain — the row in the opening table — cost $0.003780, which is 95 % more for four identical calls, because every step now carries everything before it. It produced the dangerous output. Fluent, citing its arithmetic, correct on every number it mentions, and telling a customer nothing is owed when EUR 52.08 is owed.

The difference between the two is one ternary. A chain that carries less produces answers that are obviously incomplete; a chain that carries everything produces answers that are confidently wrong — and only the second kind gets sent.

Neither is the real failure. The real failure is that the pipeline decided, before reading anything, that this task is four steps over the contents of an email. Nowhere in that structure is there a place to say "the registration country is not in this email; go and get it". Chaining is right when the decomposition is known in advance and stable. Here it was a guess, and the guess shipped.

Routing "classifies an input and directs it to a specialized followup task".¹ The name is new; the mechanism is the dispatcher, older than almost everything else in this book. What is new is that the classifier can be a model — which is what makes it fail in ways a switch never did.

const answer = await route(email,
  (q) => classifyWithSmallModel(q),          // cheap model, one call
  { billing: billingAgent, tax: taxAgent, dunning: dunningAgent },
  taxAgent,                                  // deterministic, chosen in advance
);

Two things about that last argument. It is not error handling; it is the pattern. A model-based router has a failure mode a dispatcher does not: it can return a label that does not exist, time out, or — the expensive one — return a plausible wrong label with no signal that it is wrong. All three have to land somewhere, and the somewhere cannot be another model call, because you are already in the branch where model calls failed.

The second thing is that the router's own prompt is not free. To choose a model, a router needs a catalogue of models to choose from, and every entry in it is input the router pays for before it has read the user's question. At the input rate this course prices with, a catalogue of about 3,800 tokens already costs as much as the entire five-call agent run in the opening table. In practice the routing call runs on a cheap model, which is the whole reason routing pays for itself; but the arithmetic is worth doing in that direction rather than assuming. Routing is wrong precisely when the routed task is cheaper than the routing decision.

Anthropic splits this one in two: sectioning — "breaking a task into independent subtasks run in parallel" — and voting — "running the same task multiple times to get diverse outputs".¹ They share a diagram and share almost nothing else.

Sectioning is the cheap win, and it is the line from patterns.ts: three specialists — billing, tax, policy — each with its own window and tools, over the same email, one synthesis call at the end. Identical work, ordered two ways:

Same token for token, 1.8 times faster. That is why the pattern earns its own name: it is the only one of the five that improves something without costing anything. The catch is that the sections must be genuinely independent — give section B a fact section A produces and Promise.all runs them both against a state that does not exist yet. The for loop hid that bug; the one-liner exposes it.

Voting is a different animal wearing the same picture. Running the same question k times and taking the majority is self-consistency, published by Wang et al. in March 2022 as a decoding strategy, almost three years before anyone called it an orchestration pattern. Its abstract is precise about the mechanism — "first samples a diverse set of reasoning paths instead of only taking the greedy one, and then selects the most consistent answer by marginalizing out the sampled reasoning paths" — and about the gain: +17.9 points on GSM8K.²

Two things follow that the picture hides. First, voting requires the sampling of Chapter 17: at temperature zero all k samples are the same sample, and the majority is one answer paid for k times. Second, it only works where a majority is meaningful — on the invoice reply above there is nothing to count, because five drafts are five different sentences. Voting is for tasks with a short, comparable answer, which is Wang's benchmarks exactly and almost nothing a customer-facing agent does.

Measured here on 20 three-step word problems whose answers are computed rather than judged, with the local model of Chapter 23 reasoning step by step:

Five times the calls, five times the tokens, exactly five times the bill, and not one additional correct answer. Voting is a bet, not an improvement, and this run lost it.

Two caveats, before anyone quotes that as a refutation of Wang. Twenty trials cannot distinguish 45 % from 60 % — the interval is the width of the claim, which is Chapter 4's discipline turned on my own result. And the published gains come from models orders of magnitude larger, where the diverse reasoning paths voting marginalises over are actually diverse. What transfers is not the number: it is that the multiplier is exact and known in advance while the gain is neither.

In the orchestrator-workers workflow "a central LLM dynamically breaks down tasks, delegates them to worker LLMs, and synthesizes their results", and the difference from sectioning is that "subtasks aren't pre-defined, but determined by the orchestrator".¹ The ancestry here is not from language models at all: this is master-worker, and the version where workers write findings into a shared space that a controller reads is the blackboard architecture, from speech understanding research in the 1970s. What is new in 2026 is that the controller is a model and therefore the decomposition can be decided per input — which is the flexibility, and the cost, in one sentence.

It cost 12 model calls against the single agent's 5, and it reached the same verdict. Then it did something worth looking at closely:

orchestrator final: VERDICT=credit_note_due amount=52.08 source=worker_unverified
                  | PO_MISMATCH=yes source=worker_unverified
single agent:       VERDICT=credit_note_due amount=52.08 reason=reverse_charge_should_have_applied
                  | PO_MISMATCH=yes invoice_says=PO-4417 order_says=PO-4471

Both are right. Only one knows why. The tax worker had the invoice, the order and the tax table in its own window, reached the conclusion, and also noticed — nobody asked — that the purchase order number on the invoice does not match the order's. Then it returned a summary. The orchestrator can repeat both statements and check neither, because the evidence stayed in a window it never saw. That is Chapter 24's closing question, answered: the parent gets to look at whatever the child chose to write down.

The fix is a flag, and it has a price:

Twelve per cent more input tokens, 8.3 % more money, and the phrase source=worker_unverified disappears from the answer. That is the trade in every multi-agent system and it is almost never stated: the child's clean window is worth having, the parent's ability to audit it is worth paying for, and you cannot have both for free.

So when is orchestrator-workers wrong? Here, on this task. It bought a correct answer that one agent with the same four tools also reached, for 1.66 times the cost and 2.3 times the wall clock, and it made that answer harder to defend. Anthropic's own guidance says as much before the patterns start: find "the simplest solution possible, and only increasing complexity when needed", because "agentic systems often trade latency and cost for better task performance".¹ The tables above are that sentence with numbers under it.

One call generates, another evaluates, and the loop repeats until the evaluation passes.¹ The published ancestors are Self-Refine — the same model as "generator, refiner, and feedback provider", reporting about 20 points of absolute improvement averaged over seven tasks³ — and Reflexion, which stores the critique in an episodic buffer across attempts and reports 91 % pass@1 on HumanEval where the baseline reached 80 %.⁴

The cost model is the simplest of the five: two calls per round, and the round count is not yours. Three rounds of refinement on a task that took one call is six calls, so the pattern's floor is 6× and its ceiling is whatever cap you set — which makes the budget exit of Chapter 23 mandatory rather than tidy.

The ceiling is subtler, and it is measurable. On the same 20 problems the local model answered 9 correctly. Then it was shown each of those answers and asked whether it was right — without being told the answer was its own, which removes the flattery confound and leaves the capability one:

That is a better judge than the section title implies, and saying so is the point of measuring rather than asserting: it blocked nothing correct and caught 8 of 11 mistakes. As a filter, it is worth its calls.

As a stopping rule, which is what an evaluator-optimiser loop actually uses it for, those three approvals are the whole story: they end the loop with a wrong answer in hand, and no number of extra rounds ever reaches them. A refinement loop cannot become more correct than its judge. Buying more rounds buys attempts at the errors the judge can see, at full price, and nothing at all against the ones it cannot.

Hence the rule: an evaluator earns its calls only when it has something the generator does not. A compiler, a test suite, a schema validator, a different model, a human. Self-Refine's own results are measured against human preference and task metrics, never against the model's opinion of itself. If your evaluator's only advantage is a different prompt, you are paying double for agreement. Chapter 29 builds the version with a real advantage: a golden set with the answers written down in advance.

The five above are shapes for your code. Below them sits a second family that is often listed alongside them and should not be: ReAct, Reflexion, plan-and-execute and tree of thoughts are reasoning loops, and their cost is in requests.

Chapter 12 was about reasoning inside the model, which you pay for in output tokens on one call. This is the other kind. The difference matters when the bill arrives: a longer chain of thought makes one call more expensive, and a reasoning loop makes one task into many calls, each of which resends everything before it — the quadratic Chapter 23 measured in its runaway table.

The tree-of-thoughts paper publishes its own cost table, which is rarer than it should be. On Game of 24 with GPT-4: input/output prompting best-of-100 solved 33 % at $0.13 per case, chain of thought best-of-100 solved 49 % at $0.47, and tree of thoughts solved 74 % at $0.74, with the authors noting it "could require 5-100 times more generated tokens than CoT".⁷

Nearly six times the cheap method's price for a bit more than double the success rate. Whether that is a bargain depends on what a failed case costs you — the question to ask before adopting any of these four.

This course does not reimplement them. All four have reference implementations by their own authors, in Python, and their value is being the source rather than a translation: ysymyth/ReAct, noahshinn/reflexion, princeton-nlp/tree-of-thought-llm and AGI-Edgerunners/Plan-and-Solve-Prompting. Read the prompts in those repositories; the prompts are the papers.

Now multi-agent proper, where most of the confusion lives. There are two ways for one agent to involve another, they are not variants, and the difference is who is in charge afterwards.

Agent as a tool. The parent calls it, gets an answer, and continues. It is the Chapter 18 tool interface with a whole agent behind it, and the parent never loses control. This is what the orchestrator above does.

Handoff. The parent transfers the conversation and does not get it back. OpenAI's guide is the clearest published statement: handoffs are "a one way transfer that allow an agent to delegate to another agent... If an agent calls a handoff function, we immediately start execution on that new agent that was handed off to while also transferring the latest conversation state."⁸

A vocabulary warning, because this trips people constantly: "handoff" is one SDK's word, not a standard. It is terminology from the OpenAI Agents SDK and that guide, which also names the two arrangements "manager" and "decentralized" and notes that in the manager pattern "edges represent tool calls whereas in the decentralized pattern, edges represent handoffs".⁸ There is an open standard in this space — A2A, at version 1.0.0, under the Linux Foundation's copyright, with a versioned release history and a documented list of breaking changes, whose stated principle is opaque execution: agents "collaborate based on declared capabilities and exchanged information, without needing to share their internal thoughts, plans, or tool implementations".⁹ That is not a handoff, and the comparison belongs in Chapter 26. What matters here is that one of the two words is a library's API and the other is a specification with governance.

The distinction is a data structure, not a diagram:

export type EdgeKind = "tool" | "handoff";
export interface AgentEdge { from: string; to: string; kind: EdgeKind }
export interface AgentGraph { root: string; agents: Record<string, AgentSpec>; edges: AgentEdge[] }

/** One agent may not be both a tool of X and a handoff target of X. */
export function conflicts(g: AgentGraph): AgentEdge[] {
  const seen = new Map<string, EdgeKind>();
  const bad: AgentEdge[] = [];
  for (const e of g.edges) {
    const key = `${e.from}->${e.to}`;
    const other = seen.get(key);
    if (other && other !== e.kind) bad.push(e);                            
    else seen.set(key, e.kind);
  }
  return bad;
}

/** Every agent reachable from the root, and at what depth. */
export function reachable(g: AgentGraph): Map<string, number> {
  const depth = new Map([[g.root, 0]]);
  const queue = [g.root];
  while (queue.length) {
    const id = queue.shift()!;
    for (const e of g.edges.filter((x) => x.from === id)) {
      if (depth.has(e.to)) continue;
      depth.set(e.to, depth.get(id)! + 1);
      queue.push(e.to);
    }
  }
  return depth;
}

Twenty lines, two bugs you would otherwise find in production. reachable finds the agent nobody can get to — configured, paid for, never called. conflicts refuses the edge that is both kinds at once, which sounds pedantic until you read it aloud: the parent both keeps control and gives it away. Run it on a five-agent system with one orphan and one double edge:

reachable: lead@0 billing@1 tax@1 dunning@1
orphans:   ghost
conflicts: lead->tax

Now the measurement this section exists for, and the only one in the chapter taken against a real model rather than a scripted one.

A customer states a constraint in their first message — our account is registered in Portugal, not Spain; everything tax-related has to use Portugal — chats about something else, then asks a question billing must answer. The case is transferred. Twenty-four trials, a different country and company each time, four transfer payloads, and the receiving agent is then asked one question: in which country is this customer's account registered?

The third row is a control and behaves like one: the fact is not there, so it cannot be recalled. The other three are the finding.

The full transcript is 173 tokens and works 83 % of the time, its four failures being Chapter 24's subject rather than this one's. The typed record is 69 tokens — seven more than the summary — and works every time, because the constraint sits in a named field instead of a sentence.

And the summary is the row to stare at. It failed 24 times out of 24, and the reason is not that the reader missed it. The constraint appeared in only 1 of the 24 summaries at all. The receiving agent was not careless; it was handed a text that did not contain the answer. A summary is a compaction you did not write, produced by a model whose window you cannot see, optimised for reading like a summary — and "the customer says our records have the wrong country" is exactly the kind of clause a summariser drops as procedural noise.

An honest limit on that number: the summariser is a half-billion-parameter model and a larger one would keep more. What does not improve with size is the shape of the risk — the sending agent decides, per handoff, per phrasing, unobservably, which facts survive. The typed record does not depend on that judgement at all, which is why it wins by construction rather than by intelligence. Whatever must survive a transfer should be a field, not a sentence.

The same reasoning applies in the other direction, to the agent-as-tool topology, and the earlier table already priced it: what comes back from a worker is a summary too, and paying 8.3 % more to receive the evidence with it is the same fix seen from the parent's side.

Three closing facts, all from tables above.

A multi-agent system multiplies calls, and calls are quadratic in context. The orchestrator made 12 model calls where one agent made 5, and each carries its own growing transcript — 3,628 input tokens against 2,697, a gap that widens with the length of the task.

Every boundary is a lossy channel. Two agents means one summary. Four agents in a chain means three, composed, each written by a model optimising for something other than your decision.

The single agent found something nobody asked for. The purchase-order mismatch surfaced because one window held the invoice and the order at once. Splitting work across specialists also splits the ability to notice that two facts disagree.

None of which argues against the published multi-agent frameworks, worth reading as primary sources rather than through tutorials.¹⁰ It argues for making the second agent earn its place.

So, a test rather than a preference. Add a second agent when at least one of these is true: the sub-task needs a clean window the parent must not inherit (Chapter 24); the sub-tasks are genuinely independent and the wall clock matters, which is the 1.8× above; the sub-task needs different permissions or a different model, which Chapter 30 turns into a security argument; or the sub-task is owned by someone else, which is where a real protocol starts to matter. If the answer is "so each agent has a clearer prompt", give the one agent a clearer prompt. It is free.

You can now name the five patterns, price them against each other on one task, tell an orchestrator from a sectioner and a tool call from a handoff, and defend a single agent with a table instead of a preference.

Every arrangement here shared one convenience that will not survive contact with anything real: all the tools belonged to us. Invoice, order, tax table, the workers behind the orchestrator — same repository, same deploy, same types, same people.

Now put one of them on the other side of a company boundary. The tax table belongs to an accounting vendor, the order record to a warehouse system, and neither has read your Tool interface. You need a way for a model you did not write to discover, describe and call a capability someone else operates — with authentication (which is Chapter 27's half), versioning, and the guarantee that a server cannot read the rest of your conversation. That is a protocol problem, it has a specification with a normative schema, and almost everything indexed about it describes a revision that no longer exists.

Chapter 26 reads that specification instead of summarising it, and starts by typing JSON-RPC into a terminal by hand.

Every cost and token count above came from the scripted provider described in the second section, on Node 22 over a loopback interface, counting with the o200k_base encoding and priced at the rates Chapter 16 read on 6 September 2026 — $2.00 per million input tokens and $12.00 per million output. Wall-clock figures are from the same runs with the provider's latency set to 400 ms per call and tools to 50 ms, so they measure the arrangement rather than any provider. The two real-model measurements — the handoff table and the voting-and-judging table — used Qwen/Qwen2.5-0.5B-Instruct in float32 on the CPU behind an endpoint of the same shape, greedy except where a temperature is stated, with intervals computed by Wilson's method from Chapter 4. No request in this chapter went to a paid endpoint, and no number in it was estimated.