How to assemble the model's working view without drowning it in irrelevant information or mixing data with authority.
◷ Estimated reading time: 5 min
Context Is the Model's Runtime View
The model does not see your database, repository, or organization directly. It only sees the context assembled for the current call. Context engineering is therefore the architecture of the model's temporary working world.
Context may include instructions, conversation history, retrieved knowledge, examples, tool definitions, tool results, working state, and output constraints. Each item consumes attention and tokens, so selection is an architectural decision.
Static context provides consistency. Dynamic context provides specificity. A strong system keeps the static layer compact and retrieves the rest on demand.
Context Rot and Attention Dilution
A large context window is capacity, not a guarantee of equal attention. Irrelevant history, duplicated instructions, stale tool results, and oversized document dumps can make the model miss the important signal.
Context rotAs a conversation or agent trajectory grows, outdated assumptions and repeated observations accumulate. The system becomes slower and less reliable even though nothing technically exceeds the context limit.
Common controls include summarizing old turns, checkpointing state outside the prompt, retrieving only relevant chunks, clearing obsolete tool results, and starting a fresh model call at stable boundaries.
Context Budgeting
Allocate the context window deliberately. A practical budget may reserve space for:
stable instructions and policies
the user's current request
retrieved evidence
working state and recent observations
tool schemas that are actually eligible
enough output space to finish the task
Do not fill the input to the maximum and leave too little room for the answer or tool-use loop.
Compression, Caching, and Freshness
Compression reduces repeated text while preserving decision-relevant facts. Caching can reduce latency and cost for stable prompt prefixes, but caching does not make information correct or current. Every context item should have a freshness and ownership model.
Ask before adding context: Does this information change the likely decision? Is it current? Is it trusted? Can it be retrieved later? What will be removed to make room?
Key Takeaways
Context is the model's temporary runtime view, assembled by the application.
Use compact static context for stable rules and dynamic loading for task-specific knowledge and capabilities.
Large windows do not eliminate context rot, stale state, or attention dilution.
Budget, compress, cache, and expire context based on relevance and freshness.
An agent has 80 tool schemas but each task uses at most three. What is the best context strategy?
What does prompt caching solve?
A long-running task repeatedly reasons from an obsolete tool result. Which control most directly helps?