What an AI Agent Is: Five Classic Types, Two Rival Definitions

The oldest example in the field is a vacuum cleaner in a world of two squares, A and B, each either clean or dirty.¹ It survives in every textbook because it is the smallest world in which an agent can be right or wrong.

The percept is a pair — where I am, and whether it is dirty here — and the actions are SUCK, LEFT and RIGHT. The whole program is one line.

type Percept = { dirty: boolean; where?: "A" | "B" };
type Action = "SUCK" | "LEFT" | "RIGHT";

const textbook = (p: Percept): Action =>
  p.dirty ? "SUCK" : p.where === "A" ? "RIGHT" : "LEFT";   

Run it against every starting configuration of the two-square world:

A dirty, B dirty, start A    -> steps=3 clean=true
A clean, B dirty, start A    -> steps=2 clean=true
A dirty, B clean, start B    -> steps=2 clean=true

That is a simple reflex agent: it acts on the current percept alone, with no memory of anything before it. Not a toy category — a thermostat is one, and so is a single call to a language model with no conversation attached.

Now break it the way reality does. A real vacuum robot has a dirt sensor and a bumper, not a square labelled A under the carpet. Take the location out of the percept and change nothing else:

const dirtOnly = (p: Percept): Action => (p.dirty ? "SUCK" : "RIGHT");

A dirty, B dirty, start A    -> steps=3   clean=true   still dirty=0
      t=0 at=A percept={dirty:true}  -> SUCK
      t=1 at=A percept={dirty:false} -> RIGHT
      t=2 at=B percept={dirty:true}  -> SUCK

A dirty, B clean, start B    -> steps=500 clean=false  still dirty=1
      t=0 at=B percept={dirty:false} -> RIGHT
      t=1 at=B percept={dirty:false} -> RIGHT
      t=2 at=B percept={dirty:false} -> RIGHT
      t=3 at=B percept={dirty:false} -> RIGHT

The same program, two squares. From one starting state it finishes in three steps; from another it drives into the right-hand wall five hundred times and would keep going until the battery died. It cannot perceive the difference between the two situations, so it cannot act differently in them. Russell and Norvig state the general result in one line: infinite loops are often unavoidable for simple reflex agents in partially observable environments.¹

There is a fix that costs one line and no memory, worth measuring before we reach for anything cleverer.

let seed = 12345;
const rnd = () => ((seed = (seed * 1103515245 + 12345) & 0x7fffffff) / 0x7fffffff);

const coin = (p: Percept): Action => (p.dirty ? "SUCK" : rnd() < 0.5 ? "LEFT" : "RIGHT");  

Two thousand runs of an all-dirty corridor at three sizes, one seeded generator throughout:

Randomisation removes the loop entirely. It also costs: eight rooms need fifteen moves if you know what you are doing, and this agent averages 68.7 and once took 306. That is the whole chapter in miniature. Every capability we add buys correctness in a case the previous agent could not handle, and charges for it in a currency you have to name first.

An agent perceives its environment through sensors and acts through actuators. The agent program is the function from percepts to actions — every listing above is one. The percept sequence is everything perceived so far, and a simple reflex agent ignores all of it but the last item.

Rationality is the word most articles get wrong, and getting it right makes the rest of this chapter usable. An agent is not rational or irrational in itself. Russell and Norvig define a rational agent as one that, for each possible percept sequence, selects the action expected to maximise its performance measure, given the evidence of that sequence and whatever built-in knowledge it has.¹ The performance measure is not inside the agent: it belongs to the designer, and rationality is only defined relative to it.

The specification is conventionally written as four things, PEAS: performance measure, environment, actuators, sensors.

Notice which row is the odd one out. Almost every team building agents in 2026 writes down E, A and S — the tool schemas, the integrations, the message format — because the code will not run without them. Almost nobody writes down P. Without it, "our agent is doing well" has no meaning anyone can check, and "rational" cannot be applied to the system at all, only to a demonstration. Chapter 29 is about turning P into a number, and this is why it exists.

    ┌───────────────────────── the environment ─────────────────────────┐
    │                                                                   │
    │   ┌──────────────────────── the agent ─────────────────────┐      │
    │   │                                                        │      │
 ───┼──►│  sensors  ──►  the agent program  ──►  actuators  ─────┼──────┼──►
percept │                                                        │    action
    │   └────────────────────────────────────────────────────────┘      │
    └───────────────────────────────────────────────────────────────────┘
                                   ▲
              the performance measure lives out here, in the head of
              whoever built the thing, and the agent cannot change it

Task environments are further classified along seven axes, five of which decide most of the difficulty here: fully or partially observable, deterministic or not, episodic or sequential, static or dynamic, known or unknown.¹ An agent talking to real tools over a real network is in the hard corner of all five — non-deterministic even at temperature zero (Chapter 17), and, the underestimated one, unknown, because you have no reliable model of what your own tools do to the world. That is why Chapter 23's loop needs error handling more than planning.

Real floors are not one-dimensional, so promote the world to a plan. Hash marks are walls, asterisks are dirt, and the robot starts in the middle chamber:

        col  0 1 2 3 4 5 6
      row 0  * . . # . . *
      row 1  . # . # . # .
      row 2  . # . S . # .        S = the robot starts here
      row 3  . # . # . # .
      row 4  * . . # . . *

The obvious upgrade is memory. The agent keeps a map: every square it has stood on and every square where the bumper fired. Its rule is to walk into an adjacent square it has not visited — right, then down, then left, then up — and back off when everything around it is known. This is a model-based reflex agent: it maintains internal state from the percept history, so it can act on what it cannot currently see.

It is a real improvement, and still not enough:

5,000 steps allowed -> steps=5,000  distinct squares visited=13/25  still dirty=2/4

Five thousand moves, half the floor never seen. The map is correct and the rules are correct. What the agent cannot do is use the map to go somewhere: its rules only ever answer "which of my four neighbours should I step into", so once it runs out of unvisited squares next to it, it has no way to express the thought there is an unvisited square eight moves away and I would like to be standing on it. It knows where it is. It does not know where it wants to be.

A goal-based agent holds, on top of its model of the world, a description of the situation it wants to bring about, and chooses actions by searching over sequences of them until it finds one that ends there. Goals turn action selection from a lookup into a search.

The goal is "no dirty square remains". The search is a breadth-first walk to the nearest dirty square, and the path it returns is the plan.

goal-based (fewest moves)      -> moves=27  battery=52  still dirty=0
      from 2,3 -> 4,6 via 5 moves:  2,3 2,4 3,4 4,4 4,5 4,6
      from 4,6 -> 0,6 via 4 moves:  4,6 3,6 2,6 1,6 0,6
      from 0,6 -> 4,0 via 10 moves: 0,6 0,5 0,4 1,4 2,4 2,3 2,2 3,2 4,2 4,1 4,0
      from 4,0 -> 0,0 via 4 moves:  4,0 3,0 2,0 1,0 0,0

Twenty-seven moves, floor clean. But look at the battery column and the last leg of the plan. Column 0 is carpeted: crossing a carpeted square costs six units of battery, a tiled square one. The agent went home up column 0 because that is four moves instead of eight, and those four carpeted moves cost 24 where the eight-move detour would have cost 13.

It cannot do otherwise. A goal is a binary test: the floor is clean or it is not. Every plan that ends with a clean floor satisfies it equally, so when several succeed the agent has nothing to choose between them. Preferring one success over another needs a number over outcomes, and that number is a utility function. An agent that maximises it is a utility-based agent.

The change to the code is one term inside the search. Breadth-first search counts moves; make it count cost instead and you have Dijkstra's algorithm and a different agent:

const nd = dist.get(k)! + (byCost ? cell.cost : 1);   // <- the entire difference

goal-based    (fewest moves)   -> moves=27  battery=52  still dirty=0
utility-based (cheapest route) -> moves=31  battery=41  still dirty=0
      from 4,0 -> 0,0 via 8 moves: 4,0 4,1 4,2 3,2 2,2 1,2 0,2 0,1 0,0

Four extra moves, eleven fewer units of battery: twenty-one per cent cheaper. Same goal, same map, same code but for one term. The two agents differ only in what they are trying to be good at, and they take different routes home.

This is also the first point where the agent needs something it cannot produce. Someone has to decide what a unit of battery is worth relative to a move. Utility is the performance measure written in a form the agent can compute with, and writing it is the designer's job. When people say an agent "optimised the wrong thing" they almost never mean a bug. They mean this line was written carelessly.

Now let dirt come back. Four rooms get dirty again at four different rates, and the agent is never told them. It visits one room per tick and sees only that room. The performance measure is room-ticks spent dirty over 4,000 ticks — lower is better.

A learning agent, in the textbook's decomposition, is any of the above plus three parts: a learning element that changes the agent, a critic that tells it how the agent is doing against a fixed performance standard, and a problem generator that proposes actions worth trying for what they would teach.¹ Three policies in the same environment. The first does not learn; the second and third learn the same thing and use it differently.

The hidden rates were 0.35 for the kitchen, 0.05 for the hall, 0.02 for the study and 0.01 for the attic — and learner A found them. It correctly identified the kitchen as the dirtiest room in the house, then went to the kitchen every tick for the rest of the simulation while the other three sat dirty forever. It is five times worse than not learning at all, and it is not broken.

The lesson is the utility section's. Learner A maximised "probability that the room I am about to visit is dirty". The performance measure was "room-ticks spent dirty". Different numbers; the second is what the critic was scoring, and nobody told the agent. Learner B multiplies the same learned rate by the time since the last visit — the dirt it expects to find rather than the chance of finding any — and beats the patrol it started from.

One implementation detail decided the result. In the first version of learner B, a room where no dirt had turned up in three visits got a rate of exactly zero — and zero times anything is zero, so it was never visited again and the estimate could never be corrected. Smoothing the fraction, successes plus one over trials plus two, turned 11,895 into 1,576. "Not observed yet" and "measured and came out zero" are different claims, and a system that stores them in the same field makes decisions it cannot undo.

  1  simple reflex    percept ────────────────────────────────► rules ────► action
  2  model-based      percept ──► [state] ──────────────────► rules ────► action
  3  goal-based       percept ──► [state] ──► [goal] ──────► search ───► action
  4  utility-based    percept ──► [state] ──► [goal] ──► [U] ──► argmax ► action
  5  learning         all of the above, plus [critic] ──► changes the parts above

Every one of the five is in production today under another name.

Two rows are closer than an analogy, in a way that costs money.

The chat is a model-based reflex agent whose model is not internal. In the textbook the state is a variable inside the agent program. In a chat it is the transcript: it lives on your side, is re-sent in full on every call, and is rebuilt from scratch inside the model each time. That is Chapter 16's quadratic bill, and it is the same object the textbook drew as a box labelled "state". Here is the difference, measured on one follow-up question with and without the two messages before it:

with the transcript      prompt=67  "The current temperature in Lisbon, Portugal is 15°C."
without the transcript   prompt=29  "Lisbon is the capital of Portugal, not a city in Portugal."

The same model, the same three words of user input, and the second one is the corridor robot driving into the wall. There were no tools in that run, so the 15 is invented — but the state is what makes the follow-up mean anything at all. You rebuild it every time and pay 2.3× the input tokens for it on a two-turn conversation. Chapter 16 measured what that multiplier reaches by turn forty.

Reflexion is a learning agent that changes its input rather than its program. In the textbook decomposition the learning element modifies the performance element. Reflexion leaves the weights alone and writes reflective text into an episodic buffer that the next attempt reads.³ The learning element is a prompt, the memory a database row, the performance element a frozen model — and the diagram is the textbook's, unchanged.

And here is the honest limit of the mapping. The five types classify the agent program. In 2026 that program is split down the middle: some of it is your code, some of it is inside weights you did not train. When a model decides on its own to call a tool, is the goal test in your program or in the model? The taxonomy has no answer, because when it was written there was nowhere else for it to be — and that question is exactly where the two modern definitions part company.

The definitions are arguments about behaviour, and much easier to judge with a trace in front of you.

The loop below sends the conversation to a model; if the reply contains a tool call it executes the tool, appends the result and sends the whole thing again. It runs against a local Qwen2.5-0.5B-Instruct behind an OpenAI-shaped endpoint on this machine — the seam from Chapter 14, so the loop neither knows nor cares what is behind the port.

const BASE = process.env.LLM_BASE_URL ?? "http://127.0.0.1:8799/v1";

async function loop(question: string, maxTurns = 6) {
  const messages: Msg[] = [
    { role: "system", content: SYSTEM },
    { role: "user", content: question },
  ];

  for (let turn = 1; turn <= maxTurns; turn++) {
    const reply = await call(messages, TOOLS);
    const calls = reply.choices[0].message.tool_calls ?? [];
    messages.push(reply.choices[0].message);

    if (!calls.length) return messages;                      

    for (const c of calls) {
      const out = runTool(c.function.name, JSON.parse(c.function.arguments));
      messages.push({ role: "tool", name: c.function.name, content: out });
    }
  }
  throw new Error("turn cap reached");                       
}

Two lines carry the whole idea, and both are marked; the rest is bookkeeping. All three behaviours are visible in one run. Asked something it can do itself, the model answers. Asked something it cannot, it calls:

=== a question the model cannot answer, one tool available
  turn 1  prompt= 187  out= 21  finish=tool_calls  CALL get_temperature({"city": "Oslo"})
          tool  get_temperature -> {"city":"Oslo","celsius":4}
  turn 2  prompt= 238  out= 12  finish=stop        TEXT "The current temperature in Oslo is 4
                                                        degrees Celsius."
  => model calls=2  prompt tokens=425  output=33  wall=6,257 ms
  => stopped by: the model produced text instead of a call

And it stops — the third behaviour, and the easiest to miss, because it looks like nothing happening. The loop ends because turn 2 came back without a tool call. Nobody decided that; the model did, by emitting prose. The termination condition of this program is the sign of an absence.

Two more runs are worth the space. Asked to compare two cities, the model issues both tool calls in one turn, gets both readings back, and gets the comparison wrong:

  turn 1  prompt= 188  out= 43  finish=tool_calls  CALL get_temperature({"city": "Oslo"}),
                                                        get_temperature({"city": "Lisbon"})
          tool  get_temperature -> {"city":"Oslo","celsius":4}
          tool  get_temperature -> {"city":"Lisbon","celsius":19}
  turn 2  prompt= 284  out= 13  finish=stop        TEXT "Oslo is currently warmer than Lisbon
                                                        at 4°C."

The tools worked. The parallel call worked. The loop worked. The answer is false, with both correct numbers sitting in the transcript. Wrapping a model in a loop does not make it reason; it gives a model that is wrong the ability to act on being wrong — which is Chapter 30 in advance, and half of Chapter 29.

Now delete the marked return and let the loop run to its cap instead. Same question, same model:

  turn 1  prompt= 187  out= 21  CALL get_temperature({"city": "Oslo"})
  turn 2  prompt= 238  out= 12  TEXT "The current temperature in Oslo is 4 degrees Celsius."
  turn 3  prompt= 261  out= 30  TEXT "Could you please specify the exact location you're..."
  turn 4  prompt= 302  out= 14  TEXT "Sure! Could you tell me which city you're interested in?"
  turn 5  prompt= 327  out= 35  TEXT "I'm sorry, but I need more details to provide an..."
  turn 6  prompt= 373  out= 12  TEXT "Which city would you like to know the temperature for?"
  => model calls=6  prompt tokens=1,688  output=124  wall=25,261 ms  stopped by: turn cap

Four times the input tokens, four times the wall clock, and an ending in which the agent has forgotten what it was asked and is interrogating the user about a question they answered on turn one. The correct answer was on screen at turn 2, and every turn after it made the transcript worse.

So an agent is not a loop. It is a loop plus a rule for leaving it, and this one has exactly one such rule. Chapter 23 finds five, and shows what breaks when each is missing.

Both quoted rather than paraphrased, because the paraphrases are where the confusion is manufactured.

Definition one puts the boundary at who controls the flow. Anthropic's Building effective agents names the ambiguity and rules on it:

"At Anthropic, we categorize all these variations as agentic systems, but draw an important architectural distinction between workflows and agents: Workflows are systems where LLMs and tools are orchestrated through predefined code paths. Agents, on the other hand, are systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks."⁴

The test is a question about your source code: who chose the next step? A switch in your program: workflow. The model: agent. The same document says agents "are typically just LLMs using tools based on environmental feedback in a loop" — which is exactly the listing above.

Definition two puts the boundary at independence from the user. OpenAI's A practical guide to building agents opens its definitional page like this:

"While conventional software enables users to streamline and automate workflows, agents are able to perform the same workflows on the users' behalf with a high degree of independence. Agents are systems that independently accomplish tasks on your behalf."⁵

Two sentences later, on the same page, it excludes:

"Applications that integrate LLMs but don't use them to control workflow execution—think simple chatbots, single-turn LLMs, or sentiment classifiers—are not agents."⁵

Read those quotations in order. The opening sentences draw the line at independence: does this thing go off and finish the job without me? The fourth draws it at control of execution, which is Anthropic's line exactly. Different tests, same page, and there are real systems on which they disagree.

There is a vocabulary collision underneath, and it causes arguments in real meetings. In the first document a workflow is an architecture, and it is the thing that is not an agent. In the second a workflow is "a sequence of steps that must be executed to meet the user's goal" — the job itself, which every agent has one of. "We replaced the workflow with an agent" is coherent under the first definition and close to meaningless under the second.

Three systems that exist in 2026, under both definitions.

Two of the three change sides. That is not a failure of either document. It is a warning about a kind of meeting in which two people who agree completely about what a system does spend an hour disagreeing about what to call it.

The definitions collide because each collapses two independent questions into one word. Separate them and the disagreement becomes a table, which is more useful than a verdict.

Each definition disputes a different cell, and the other two are not in dispute at all. So when the label matters — in a contract, a risk review, a postmortem — the two sentences worth writing are not "is it an agent" but who chose the next step and who was watching. Both are answerable by reading code, neither needs anyone's definition, and together they carry every consequence the label was standing in for.

None of this is new. Wooldridge and Jennings surveyed the competing senses of "agent" in 1995;⁶ Franklin and Graesser asked this chapter's question in 1996, gathered the definitions in circulation and found that they disagreed.⁷ A 2023 survey still defines agents from first principles — "artificial entities that sense their environment, make decisions, and take actions"⁸ — because nothing settled existed to cite, and CoALA describes parts rather than drawing a boundary at all.⁹ Thirty years of declining to agree says the word is doing more than one job.

Now the consequence that arrives before the philosophy, which is the bill.

Every measurement here has the same shape. The single call cost 39 input tokens; the same question with one tool cost 420 across two calls; the loop with its stopping rule removed cost 1,688 across six. The growth is worse than linear, because turn n carries every previous turn with it: the prompt column of that six-turn run reads 187, 238, 261, 302, 327, 373. Chapter 16 derived that the total is $\Theta(n^2)$ and fitted the curve on a real conversation. An agent turns every task into that conversation, whether or not a human ever sees it.

If those measured token counts had gone to a commercial endpoint at the rates Chapter 16 read on 6 September 2026 — $2.00 per million input tokens and $12.00 per million output — the four runs price out like this:

Row two against row one is the number to keep. Seven and a half times the cost, for a worse answer to a question the model already knew. Nothing was misconfigured: a tool existed, so the model used it — and Chapter 18's finding, that the price of a catalogue and not its accuracy is what hurts, has its cheapest demonstration here with a catalogue of one.

Which is why the useful half of both documents is the half about not building this. Anthropic's is blunt: find the simplest solution possible and add complexity only when needed, which "might mean not building agentic systems at all", since agentic systems "trade latency and cost for better task performance" and "for many applications, optimizing single LLM calls with retrieval and in-context examples is usually enough".⁴ Its case for an agent is narrow: open-ended problems where you cannot predict the number of steps and cannot hardcode a path, in an environment you trust, accepting "higher costs, and the potential for compounding errors".⁴ OpenAI's screen is the mirror image — complex judgement, unmaintainable rule sets, unstructured data — and ends the same way: "otherwise, a deterministic solution may suffice".⁵

So, in this chapter's taxonomy: a fixed number of steps in a fixed order is a pipeline, and calling it an agent will not make it faster. If the number of steps depends on what you find on the way, you want a loop — and you buy that flexibility with N calls, a quadratic transcript, and a system that can be wrong N times instead of once.

You now have the taxonomy, both modern definitions, the two axes that make them compatible, and a short loop that answers, calls and stops.

That loop has one way to end: the model stops asking for tools. Chapter 23 breaks it on purpose, seven times, and each break adds a piece. An impossible task, and it never ends — a turn cap. A night of running, and the bill arrives — a budget in dollars. A tool that fails — an error the model can act on. The same call twice — an idempotency key. A file it should not have touched — a human approval. A restart halfway — session persistence. A tool that takes three minutes in silence — progress and cancellation. What comes out is a harness, the file the rest of this course runs on.

Which leaves the question this chapter's disputed diagonal was really about. A loop that decides its own next step has to decide when to stop, and we have just watched what happens when it cannot: six turns, four times the bill, and an agent interrogating the user about a question it had already answered. Stopping is not one condition. How many are there, and which one fires first?

Lilian Weng's LLM Powered Autonomous Agents (2023) is the best-known decomposition of a language agent into planning, memory and tool use, and is the right next read alongside the two vendor documents; its three components are Chapters 23, 24 and 18 of this course in that order.

Every number in this chapter was produced on this machine and nothing was estimated. The corridor, the floor plan, the four agents that walk it and the three patrol policies are the TypeScript above, run on Node 22; the randomised agent's figures are means over 2,000 seeded runs each and the patrol figures are single seeded runs of 4,000 ticks. The model traces come from Qwen2.5-0.5B-Instruct in float32 on CPU with greedy decoding, served over loopback by a small local Python endpoint that loads the weights and speaks the OpenAI chat-completions shape — the seam again, with the tensors on the Python side and the loop on the TypeScript one — so the token counts are that model's tokenizer and the latencies are that machine's. The only figures taken from elsewhere are the two prices in the cost table, which are the rates Chapter 16 read from OpenAI's pricing page on 6 September 2026, applied here to locally measured token counts as an illustration and not as an observed invoice.