Few-Shot Examples vs Retrieved Context: Spend the Prompt Budget Deliberately

Few-shot examples show a model how to perform a task. Retrieved context supplies information needed for this particular request. Both consume input space, but removing one does not necessarily replace the function of the other.

When an agent's prompt grows, separate instructions, examples, active task state, and evidence before trimming. The useful question is which component changes the result, not which one is easiest to delete.

Keep examples about behavior

A formatting example can demonstrate how to represent uncertainty or cite evidence. It should not become an accidental source of customer facts. Use clearly fictional values and vary them enough that the model does not simply copy one answer pattern.

If the desired output can be specified with a schema or deterministic validation, use those controls where appropriate. More examples are not automatically the best way to enforce a structural requirement.

Retrieve facts for the current request

Supply the permitted source passages that answer the user's question. Keep them distinguishable from instructions. A sentence inside a retrieved document should not change the agent's tool permissions or override the task.

A summary can reduce volume, but preserve exact conditions, amounts, and identifiers when they are necessary to answer correctly. Keep a route back to the source for questions that need precise wording.

Test removal one component at a time

Compare the current prompt with fewer examples while holding retrieval fixed. Then compare retrieval selection while holding the behavior instructions fixed. Measure task completion, evidence support, output validity, latency, and tokens.

For illustration, removing 2,000 tokens of examples from a 10,000-token input reduces input length by 20%. It does not establish a 20% cost or latency reduction across the full workflow, because output, caching, and other calls may contribute separately.

Use a failure-driven budget

Reserve enough room for the current task and required evidence, then justify the examples that remain. Keep difficult cases in the fixture, including a changed instruction, an unsupported question, and a source containing misleading instructions.

If the conversation itself dominates the budget, test compaction fidelity before choosing a summary size.

Anthropic's context-engineering discussion offers relevant design context. The context-engineering guide connects it to application architecture. If persistent user context is the missing component, try Supermemory for that narrow role while retaining control of the prompt budget.