Context engineering for long-horizon AI agents

Taken together, Arize’s analysis, the arXiv paper on context window overflow, Redis’s production explainer, and Atlan’s harness-engineering comparison point to a practical shift in agent design. Long-running agents are being judged less by the size of the model’s context window and more by the control layer around it. Arize makes that shift concrete. It names shipped agent tools and memory/harness systems, including Pi, OpenClaw, Claude Code and Letta, as examples of harness-level context engineering, and describes an interactive simulator that shows a 200K-token window filling up.

The public details available in the cited sources are uneven. Arize gives concrete implementation numbers for Pi, OpenClaw, Claude Code and Letta. A research paper on solving context window overflow in AI agents gives a more general mechanism for handling tool outputs that can exceed any practical window. Redis’s explainer on context window overflow summarizes the production symptoms: hard API errors, silent quality degradation, tool output accumulation, and longer latency as prompts grow. Atlan’s comparison of prompt, context and harness engineering provides the useful stack metaphor: prompt engineering shapes the message, context engineering shapes what the model sees, and harness engineering shapes the whole agent environment.

The important news is not that context windows are too small. Builders already know that. The more useful point is that the cited agent systems are converging on four harness mechanisms that keep work alive after the transcript stops being a safe source of truth.

A shallow agent reads files, calls tools, appends the result, and hopes the model can cope. A harness-first agent blocks or reshapes large inputs before they reach the model.

A cleaner way to read the first set of limits is:

• Pi: file reads stop at 2,000 lines or 50KB, whichever comes first. The returned content includes a continuation hint telling the model which line range was shown and how to continue with offset and limit. OpenClaw inherits that behavior, then adds separate caps: bootstrap files are limited to 12,000 characters per file and 60,000 characters total. Tool results get another budget of 16,000 characters or 30% of the context window, whichever is smaller.

Claude Code uses a two-gate design. According to Arize, it checks a 256KB byte cap before opening a file, then token-counts the result against a 25,000-token budget after the read. Even for files under the cap, it defaults to returning 2,000 lines from the beginning, and truncates lines longer than 2,000 characters. If the model rereads the same file range and the file has not changed, Claude Code can return a stub instead of repeating the full content.

That is not just optimization. It changes the failure mode. Instead of letting one large read crowd out the task, the harness turns “read everything” into “read a controlled slice.” If the model needs more, it can ask for it. For builders designing agent harnesses from scratch, this is the first line of defense: never let raw external data become the transcript by default.

The next pattern is to treat context like a viewport, not storage.

Pi and Claude Code expose pagination through offset and limit. OpenClaw adds head/tail truncation in some places, keeping the beginning and end when the middle is less likely to matter. Arize says OpenClaw uses a 75% head / 25% tail split for oversized bootstrap files, and may keep both head and tail for tool results when the tail looks important, such as errors, closing JSON braces or summary-like keywords.

Letta goes further by making files live outside the prompt. Uploaded files are parsed, chunked and embedded into a vector store, giving the agent direct viewing, exact search and semantic search. When a file is open in context, Letta shows a managed view whose size scales with model context: 5,000 characters for 8K context, 15,000 for 32K, 25,000 for 128K, and 40,000 for 200K+. The number of simultaneously open files also scales, from 3 for small models up to 15 for very large ones, with an LRU policy evicting the least recently accessed files.

This is the same design idea behind production RAG: do not stuff the whole corpus into the prompt; retrieve the part that matters. The difference is that agent harnesses must do it continuously, across files, tool outputs, memory, and intermediate plans. The same constraint applies to RAG systems: retrieval is not only about relevance, but also about preserving enough context budget for the actual reasoning step.

Redis makes a related point: bigger context windows do not remove the need for context management. System prompts, retrieved documents, conversation history and tool outputs all compete for the same space. Even before a hard limit is hit, models can degrade as relevant information gets buried in long inputs.

Overflow is the obvious failure. Goal loss is quieter. The agent still has room to respond, but it forgets the original objective, misses a constraint, or starts optimizing a local subtask.

That is where compaction matters. Done badly, summarization replaces a messy but faithful history with a neat but lossy story. Done well, it preserves the task state, recent work, pending items and tool-call integrity.

Arize reports that Pi triggers compaction when estimated context tokens exceed the context window minus reserve tokens, with a default reserve of 16,384 tokens. It keeps the most recent roughly 20,000 tokens and summarizes older content into a synthetic user message prepended to the kept tail. It also avoids cutting across tool-call/tool-result pairs.

OpenClaw adds a more aggressive history policy. When history exceeds 50% of the context window, it splits messages into equal-mass token chunks, drops the oldest chunk, summarizes the dropped content through staged multi-pass summarization, and repairs tool-call/result pairing. It also performs a pre-compaction flush: a silent agentic turn gives the agent a chance to persist state to memory files before history disappears. Separately, it prunes tool results in memory with soft-trim and hard-clear behavior on a 5-minute cache TTL.

Claude Code compacts near the end of the window. Arize says its trigger is the effective context window minus a 13,000-token buffer, which puts compaction around 167K tokens for a 200K-context model. Its summarization prompt asks for structured sections covering the primary request, technical concepts, files and code, errors and fixes, problem solving, user messages, pending tasks, current work and next step. After compaction, it can reattach up to 5 recently read files within a token budget.

The pattern is clear: compaction is not “summarize the chat.” It is checkpointing. A long-running agent needs the equivalent of a save file: goal, constraints, decisions, open handles, recent evidence and next action.

Some outputs should never be placed in the context window at all.

The arXiv paper makes this concrete with a materials-science workflow. One tool generates an electronic grid structure for a molecule: a 3D matrix of dimensions 128 × 128 × 128, totaling 2,097,152 float32 elements. That output far exceeds the context window of widely used LLMs. But the next tool needs the grid as input.

The proposed solution is to store large values outside the model context and return short identifiers, or pointers. Tool wrappers inspect inputs to see whether they are raw values or memory paths. Outputs that are too large are stored in runtime memory under a path, and later tools can receive the pointer and resolve it internally. The model manipulates references, while the harness preserves the complete data. In one comparative experiment where both methods succeeded, the pointer-based approach used approximately seven times fewer tokens than the traditional workflow, according to the paper.

This is the cleanest separation between reasoning and data transport. The model does not need to “see” a 2-million-element matrix to pass it to another tool. It needs to know that the matrix exists, what it represents, and which operation should consume it next.

The same logic applies beyond scientific arrays. Large JSON responses, PDFs, logs, embeddings, media files and database exports often belong in storage, not in the prompt. For systems built around MCP tools or custom API connectors, pointer passing should be a first-class design choice, not a patch after the first overflow.

A 200K-token context window feels large until an agent starts acting. A system prompt, tool definitions, a few retrieved documents, file reads, logs, error traces and summaries can consume it faster than expected. The practical frame is not how large the window looks on paper, but how quickly agents spend it at runtime. Redis’s agent-memory guidance points toward external, durable memory for state that should survive across calls, while Atlan’s context-engineering framing separates better prompts from better context assembly. Put together, they treat the context window less like a warehouse and more like a constrained working set.

The deeper lesson is that a context window is a scarce runtime resource. Treating it as “memory” is helpful, but only if the harness behaves like an operating system: allocate, evict, page, compact, deduplicate and persist. Atlan’s layer distinction is useful here. Prompt engineering cannot fix a file reader that dumps 80,000 irrelevant tokens into the next call. Context engineering can improve the working set. Harness engineering decides whether that working set is protected in the first place.

This also changes how teams should evaluate agents. A demo prompt is not enough. Long-horizon evaluation should include growing transcripts, repeated file reads, large tool outputs, failed tool calls, resumptions after compaction, and tasks where the correct next step depends on an early constraint. Our guide to context engineering for agents covers the model-side version of that problem; the harness layer is where it becomes operational.

First, put budgets on every context source. Files, tool outputs, retrieved chunks, memory inserts and conversation history should each have explicit limits. A single global max token count is too blunt.

Second, make truncation actionable. If the harness cuts content, the model should know what range it saw and how to request more. Silent truncation is worse than rejection because it creates confident work over missing data.

Third, compact around state, not prose. Summaries should preserve the user’s goal, constraints, decisions, pending tasks, files touched, tool results that matter and the immediate next step. Tool-call pairs should stay intact.

Fourth, move large values out of the prompt. Store them, name them, and pass pointers through tools. This is especially important for agents that call APIs, process documents, or coordinate multi-agent systems.

Finally, test for goal loss separately from overflow. An agent can stay under the hard window and still drift. The right question is not only “did the API accept the prompt?” It is “does the next action still serve the original task?”

The summary below turns those patterns into a quick checklist before the FAQ.

• Long-horizon agents fail through both context overflow and goal loss, so the harness must manage more than prompt length.
• Production agent systems use hard budgets on files, tool outputs and history before raw data reaches the model.
• Pagination, search and managed views treat context as a limited viewport rather than permanent storage.
• Compaction works best as checkpointing: it preserves goals, constraints, decisions, pending work and tool-call integrity.
• Large tool outputs often belong in external storage with short pointers passed between tools instead of full values in the prompt.

This section answers the practical questions behind context engineering for long-horizon agents: what overflows, how goals get lost, and which harness patterns keep work on track.

Context overflow happens when an agent’s accumulated prompt, history, retrieved data, files and tool outputs exceed the model’s usable context window or degrade quality before the hard limit is reached.

Goal loss happens when the original task is still present somewhere in the transcript but no longer guides the agent’s next action, often after long histories or poor summarization.

They set per-source budgets, paginate file reads, retrieve only relevant views, compact history around state, deduplicate repeated reads and store large outputs outside the prompt.

Pointers let the model refer to large values stored in runtime memory, such as matrices, logs or PDFs, while downstream tools resolve the full data without placing it in the context window.

No. Larger windows help, but system prompts, tool definitions, retrieved documents, logs and history still compete for space, and relevant information can get buried before a hard limit is hit.