Context engineering: deciding what the model sees on every call
Context engineering is the practice of building, for every model call, the smallest set of information that lets the model do the job well: the instructions, the relevant examples, the right documents, the useful part of the history, the tools it needs and the results that matter. A model can only reason about what is in its context window, and it uses long, cluttered context worse than short, focused context, so what you leave out matters as much as what you put in. In agents, where the context changes on every step, it is the main thing that decides whether the agent stays on track.
Why prompts stopped being enough
A prompt is the instruction. Context is everything the model sees, and in a real application the instruction is often the smallest part.
Think about a support assistant answering one question. The model sees a system prompt with rules, maybe some examples, the definitions of five tools, the last ten messages, three chunks retrieved from the help centre, the customer's account details from a tool call, and finally the customer's new message. If the answer is wrong, the cause could be any of those: an outdated chunk, a tool result that was cut off, an old message that confused it, two rules that contradict each other. Rewording the instruction won't fix any of that.
That is why people who build agents started using the term context engineering: the job is designing the whole information flow, not only the wording.
Context is a limited resource
Every model has a context window, and even inside it, models don't use every token equally well. As context grows, attention is spread over more tokens, relevant details compete with irrelevant ones, and accuracy, cost and latency all get worse.
So treat context like memory in a small device: every token should earn its place. The question for each piece is not "could this be useful?" but "does the model need this to do this step?".
Parts of the context
| Part | What to decide |
|---|---|
| Instructions | Clear, current, no contradictions, no rules the model never needs |
| Examples | A few representative ones, chosen for this type of request |
| Retrieved knowledge | Only the chunks that answer this question, with their sources |
| Conversation history | Recent turns in full, older turns summarised |
| Tool definitions | Only the tools this step needs, with precise descriptions |
| Tool results | Trimmed to what matters, structured, not raw dumps |
| State and memory | Facts about the user or task that persist, stored outside and loaded when relevant |
| The user's request | Clearly separated from everything else |
Techniques
Select
Load information when it's needed instead of loading everything up front. Retrieval is the most common form: find the relevant chunks for this question instead of putting the whole knowledge base in the prompt. The same applies to tools and examples: an agent with 40 tools can be given only the 6 that fit the current task.
Compress
Replace long material with a shorter version that keeps what matters: summarise old conversation turns, keep the 5 relevant rows from a 500-row tool result, extract the fields you need from a long document. See context compression.
Structure
Put each kind of information in its own clearly labelled section, with sources attached to retrieved chunks. Structure helps the model find things and helps you debug what it saw.
Isolate
Split work so that each model call has a clean, focused context. A sub-agent can do a research task in its own context and return only a short summary to the main agent, so the main context doesn't fill with search results.
Persist outside the window
Keep important information in external storage, a database, a file or a memory store, and bring it back when it's needed. Coding agents do this with notes and plan files: the decisions are written down, so they don't have to survive 100 turns of conversation. See agent memory.
Order deliberately
Models tend to use information at the start and the end of the context better than in the middle. Put stable instructions first, which also helps prompt caching, documents in the middle, and the specific question or task at the end.
Context engineering in agents
Agents make all of this harder, because the context grows on every step. After 20 tool calls, the context holds the original task, 20 tool calls, 20 results and 20 rounds of reasoning, and the original instructions are now far from where the model is working.
This is why long agent runs drift. The agent forgets a constraint from the start, repeats a step, or gets confused by an old error message that no longer matters.
What helps:
- Clear old tool results once they've been used, keeping only a short note of what they showed.
- Summarise the run so far at checkpoints, and continue from the summary.
- Keep a short task state, the goal, what's done and what's next, and update it on every step.
- Give sub-tasks to sub-agents with their own clean context.
- Give the agent smaller tasks to begin with.
How to debug context
When an answer is wrong, look at the exact context the model saw before changing anything.
- Log the full context of every call, with sensitive data masked.
- For a bad answer, check: was the needed information there? Was it correct? Was it buried? Was there conflicting information?
- Fix the cause in the right place: retrieval, the tool result, the history management, or the instructions.
Most teams that do this find that their "prompt problems" are mostly retrieval and history problems.
From the floor
When I build with Claude Code, the context fills up as we keep working on a project, and it can miss some of the earlier instructions. So I give it smaller tasks, one component at a time, and check what it changed before moving forward. That is context engineering in practice: keeping each step's context small enough that the important instructions are still in view.
In the bootcamp, when we built a research agent, I kept that agent to one job, searching the web and returning a few results, because every additional task means more loops, more context, and more chances for the agent to lose track of the earlier steps.