Conversation Compaction: Test What a Summary Must Preserve

Conversation compaction should preserve the information needed for the next task while reducing prompt size. A fixed compression ratio, such as four-to-one, is not a quality target by itself. The right summary retains decisions, constraints, unresolved questions, and references without turning uncertain statements into settled facts.

Define the next task before the summary

A coding agent resuming a refactor needs the chosen approach, changed files, failed attempts, and remaining checks. A research agent needs claims, evidence, uncertainty, and open questions. These tasks require different summaries even when the original conversation is the same length.

Write a small set of questions the next session must answer. Use them as preservation tests. A summary that sounds fluent but cannot answer those questions is not an adequate handoff.

Preserve distinctions that are easy to collapse

Separate a proposed action from an approved decision, an observed result from a hypothesis, and a permanent preference from a temporary instruction. Include the source or location of important evidence so the next agent can inspect details instead of relying on an increasingly compressed paraphrase.

Keep unresolved conflicts visible. If two documents disagree about a deadline, the summary should record the disagreement rather than choose one without explanation. Compression can otherwise create false certainty that becomes harder to detect in later sessions.

Compare several budgets on the same task

For an illustrative 12,000-token conversation, a 3,000-token summary is a four-to-one reduction. That arithmetic says nothing about whether the summary preserved the only critical constraint. Compare several practical budgets and score the next task's correctness, source recovery, repeated work, and unsupported assumptions.

Test repeated compaction as well as a single summary. Information can disappear gradually when each new summary is written from the previous summary rather than the original evidence. Preserve durable records separately where the workflow requires them.

Combine compaction with selective retrieval

A compact handoff can carry the current plan while retrieval supplies detailed evidence on demand. Anthropic's context-engineering guidance discusses compaction among techniques for long-running agents; treat it as a design technique to evaluate, not a universal compression recipe.

The few-shot context-budget guide helps allocate the remaining prompt space. If you need relevant context across sessions, try Supermemory alongside a compact handoff and test which information belongs in each layer.