Effective Context Engineering for AI Agents

The core idea

Prompt engineering is about writing good instructions. Context engineering is the broader job of curating everything the model sees during a run: the system prompt, tool definitions, retrieved data, examples and the growing message history. The guiding principle is to find the smallest set of high-signal tokens that makes the desired outcome most likely.

The article explains why this matters even with large context windows. As context grows, models get worse at using all of it, an effect it calls context rot. Attention is a finite budget, so every irrelevant token has a cost.

What it recommends for each part of the context

• System prompts at the right altitude: not brittle, hard-coded if-else logic, and not vague aspirations either. Clear, direct language organised into sections, giving the minimum that fully describes the expected behaviour.
• Tools: self-contained, unambiguous and minimal, with no overlapping functions. If a human engineer cannot say which tool applies, the model will not do better.
• Examples: a few diverse, canonical examples instead of a long list of edge cases.
• Just-in-time retrieval: keep lightweight references such as file paths, queries or links, and load the data with tools when it is needed, rather than stuffing everything in up front.

Techniques for long-running tasks

When a task is longer than one context window can hold, the article describes three approaches:

• Compaction: summarise the conversation so far, keeping decisions and open issues and dropping redundant tool output, then continue from the summary.
• Structured note-taking: the agent writes notes to a file or memory outside the context and reads them back later.
• Sub-agent architectures: specialised agents do focused work in clean contexts and return a condensed summary to a coordinating agent.

Each has trade-offs, and the right choice depends on whether the task is conversational, iterative or highly parallel.

How to apply it to your own agent

• Log exactly what is in the context at each step of a failing run. Most problems become obvious once you see it.
• Cut tool output down to what the model needs, and paginate or truncate large results.
• Remove tools the agent rarely uses correctly.
• Test compaction by checking whether the agent can still complete the task from the summary alone.

Read it after Building Effective Agents and alongside Writing Effective Tools for AI Agents. Together they cover the design, the tools and the context.

Common questions

Is this different from prompt engineering? It includes it. Prompt engineering is about the instructions; context engineering covers everything else in the window too, including tools, retrieved data and history.

Do I need this if my model has a huge context window? Yes. The article's point is that quality drops as context fills, so a bigger window delays the problem rather than removing it.

Does it apply to non-Anthropic models? The advice is general and applies to any transformer-based model.

How long does it take to read? About 20 to 30 minutes, but it is worth rereading once you have an agent to apply it to.

Best for: engineers moving an agent from a demo to something that holds up on long, real tasks.