Control Context Before It Controls the Bill
Long conversations and agent runs can quietly turn context into the dominant cost. The expensive mistake is not only sending too much text once; it is repeatedly resending the same growing history on every turn.
- Context growth
- Retrieval instead of replay
- Summarisation and compaction
- File and tool-result limits
- Prompt payload observability
The situation
Agentic products accumulate history: user messages, assistant messages, tool results, source files, logs, screenshots, test failures, and generated code. If all of that is pushed into every subsequent turn, the input grows even when the actual decision only depends on a small part of the history.
The naive solution
Send the whole conversation and every available file so the model never lacks context.
The hidden constraint
More context is not free and is not always better. Repeated irrelevant history increases spend, can raise latency, and can make the model pay attention to stale or conflicting information.
The decision
Treat context as a managed resource. Every item placed into a prompt should have a reason to be there, and long-running workflows need compaction rules rather than infinite replay.
The implementation
I would separate durable state from prompt state. Keep project facts, files, and run history in storage; retrieve only what the current step needs. Summarise older conversational history, cap raw tool output, pass diffs instead of whole files where appropriate, and store large artifacts outside the context window. I would log input size by component so I can see whether system prompts, chat history, retrieved documents, files, or tool results are driving growth.
The tradeoff
Aggressive compaction can remove useful nuance. The answer is not 'always use less context'; it is to preserve the information required for correctness and remove accidental repetition.
The evidence I would look for
A healthy context strategy should stop per-turn input from growing indefinitely as a run gets longer, while validation and user outcomes remain stable.
The reusable lesson
Persistent state belongs in the product; only task-relevant state belongs in the prompt.
The consultant's questions
- What percentage of prompt input is repeated from previous turns?
- Are full files being sent when a diff or excerpt would be enough?
- Are tool results bounded before they enter model context?
- How does a 50-turn session behave compared with a five-turn session in cost and quality?