Context Engineering for Agents
A large context window is capacity, not attention. Context engineering chooses the smallest set of tokens that lets the model make the next correct decision.
Five Context Layers
- Stable contract: system rules, repository instructions, permissions, and output requirements.
- Task packet: objective, acceptance checks, constraints, and explicitly out-of-scope work.
- Working set: only the files, symbols, logs, and documentation needed for the current step.
- Recent evidence: tool results and errors, normalized into concise structured form.
- Durable state: plans, checkpoints, decisions, and artifacts stored outside the conversation and reloaded when needed.
The first two layers change slowly. The working set should change as the task moves. Durable state prevents the transcript from becoming the database.
Budget Before Retrieval
Allocate context deliberately rather than filling the advertised maximum:
| Budget | Typical content |
|---|---|
| 10–20% | rules, task, and acceptance criteria |
| 40–60% | current code or evidence |
| 10–20% | recent tool results and state summary |
| remainder | room for generation and unexpected evidence |
These are starting ranges, not protocol constants. Long output reserves and local KV-cache limits may require a much smaller input.
Selection Rules
- Search and inspect structure before opening whole files.
- Prefer exact definitions plus callers and tests over broad repository dumps.
- Return bounded tool output: counts, relevant excerpts, paths, and stable error codes.
- Keep source text for claims; label summaries as summaries.
- Remove obsolete plans and superseded errors from the active packet.
- Preserve unresolved assumptions explicitly instead of letting them vanish during compaction.
Compaction Without Amnesia
When the context grows, write a checkpoint containing objective, completed work, changed files, verification evidence, open risks, and next action. Keep identifiers and exact commands; drop conversational filler and abandoned speculation. A summary is lossy, so link it to durable artifacts rather than replacing them.
Common Failures
- Context stuffing: more files dilute the relevant dependency.
- Stale instructions: an old plan survives after the code or requirement changes.
- Unbounded logs: thousands of lines crowd out the actual failure.
- Summary laundering: an uncertain inference returns later as a fact.
- Hidden-only state: a restarted loop cannot tell what was attempted.
For small local models, context discipline often improves reliability more than extending the nominal window. Start with the native context, a narrow working set, and short tool responses; enable RoPE scaling only after measuring the task that needs it.