Memory Systems for Long-Running AI Agents

Agents that work on tasks lasting hours, such as codebase migrations or multi-step research projects, eventually generate more tokens than a model’s context window can hold. Anthropic’s engineering team frames the problem as context engineering: managing the set of tokens included when sampling from a large language model. The guidance, published at https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents, describes context as a finite resource and recommends finding the smallest set of high-signal tokens needed to reach a desired outcome.

Why Context Becomes a Constraint

The article describes a phenomenon it calls context rot, where a model’s recall accuracy declines as the token count grows. Because the window has limits, simply appending every message, tool result, and intermediate step does not scale for long-horizon work. The practical question is not how to store everything, but how to keep the most useful information available while discarding the rest.

Three Approaches to Extending Memory

Anthropic outlines three techniques for tasks that exceed the context window.

Compaction summarizes a conversation as it approaches the context limit and restarts with that summary. In Claude Code, the process preserves architectural decisions, unresolved bugs, and implementation details while discarding redundant tool outputs. The agent then continues with the compressed summary plus the five most recently accessed files. A lighter version, described as one of the safest forms of compaction, is tool result clearing, which removes raw results once a tool was called deep in the conversation history.

Structured note-taking, also called agentic memory, has the agent write notes that persist outside the context window and pull them back in later. Examples include Claude Code to-do lists and a custom agent maintaining a NOTES.md file. The article points to Claude playing Pokemon as an illustration: the agent maintained tallies across thousands of steps, built maps of explored regions, tracked achievements, and kept combat strategy notes, then read those notes back after context resets to continue sequences lasting hours.

Sub-agent architectures split work so that specialized sub-agents handle focused tasks with clean context windows while a main agent coordinates a high-level plan. A sub-agent might use tens of thousands of tokens during its work but return only a distilled summary, often one to two thousand tokens, which isolates detailed search context from the main thread.

Loading Context When It Is Needed

Rather than pre-processing all data up front, the article recommends a just-in-time approach where agents keep lightweight identifiers such as file paths, queries, and web links, then load the underlying data at runtime. This mirrors how people rely on file systems, inboxes, and bookmarks instead of memorizing everything. The article also describes progressive disclosure, where an agent discovers context through exploration, using signals like file sizes, naming conventions, and timestamps as proxies for relevance.

In practice, Anthropic recommends a hybrid strategy that combines some up-front retrieval for speed with autonomous exploration. Claude Code is given as an example: CLAUDE.md files are loaded up front, while glob and grep retrieve specific files just in time.

A File-Based Memory Tool

Alongside these patterns, the article references a memory tool released in public beta on the Claude Developer Platform with the Sonnet 4.5 launch. It is described as a file-based system that stores information outside the context window, lets an agent build a knowledge base, and helps it maintain project state across sessions. Combined with compaction, note-taking, and sub-agents, this gives developers a set of concrete options for keeping long-running agents coherent without overflowing the context window.