What is context engineering? A guide with a worked example

By Shardul ManeApril 10, 2026·10 min read

Context engineering is the work of choosing what an AI model sees on each step of a task: its instructions, its tools, the conversation so far, retrieved documents, tool results and memory. Prompt engineering is about how you word the instructions. Context engineering covers everything else that lands in the context window, and when it gets there.

Below: definitions from the people who build the models, the layers of context, why the window has to be managed at all, the main techniques, and a worked example of one support agent turn, before and after.

What is context engineering?

Start with the word context. Everything a model receives on one call (its instructions, the user's message, the conversation history, retrieved documents and tool outputs) is its context. Anthropic defines context as the set of tokens included when sampling from a model, and context engineering as the strategies "for curating and maintaining the optimal set of tokens" during inference (Anthropic, Sep 2025).

Others describe it the same way:

Andrej Karpathy, backing the term over "prompt engineering" in June 2025, called it "the delicate art and science of filling the context window" with the right information for the next step (Karpathy on X).
Google's Agent Development Kit team treats context as a first-class system with its own architecture, lifecycle and constraints, and describes what the model sees as a view compiled from richer state such as sessions, memory and files (Google Developers Blog, Dec 2025).
A survey of more than 1,400 research papers frames it as a formal discipline that goes beyond prompt design (Mei et al., 2025).

A common analogy: the model is the CPU and the context window is its RAM, the working memory it reasons over on each step. Context engineering decides what gets loaded into that RAM.

Context engineering vs prompt engineering

Prompt engineering is one part of context engineering. Anthropic calls context engineering "the natural progression of prompt engineering" (Anthropic).

	Prompt engineering	Context engineering
What you work on	The wording of instructions	Everything the model receives on each call
When it happens	Once, while you write the prompt	Every step, as the task changes
State	Usually one request	Across turns, and across sessions through memory
Sources	Text you write	Instructions, history, retrieval, tools, memory, live data
Typical failure	An unclear instruction	Missing, stale, conflicting or excess context

A clear system prompt still matters. It just doesn't help when the model never sees the customer's last ticket.

What are the layers of context?

Most agents assemble the same layers on every call:

Instructions: the system prompt, the role and the rules.
Tool definitions: what the model can call, with names, descriptions and schemas.
The user's current message.
Conversation history: the earlier turns of this session, often trimmed or summarized.
Working state: the plan, what's done, what's pending.
Retrieved knowledge: passages from documents, fetched for this question.
Memory: what was saved from earlier sessions, such as preferences, decisions and past events.
Tool outputs: results from search, databases and APIs.

What isn't a layer: the training data used to pre-train the model. That knowledge sits in the model's weights. Context engineering works with what you send at inference time, and changing the weights is training or fine-tuning, a different job.

Why does context engineering matter?

The window is finite, and models get worse at using it as it fills.

Context rot. As the token count grows, accuracy and recall degrade (Anthropic). Anthropic's engineering team compares the window to an attention budget that every new token spends a little of (Anthropic, Sep 2025).
Lost in the middle. Models use information at the start or end of a long input more reliably than information in the middle (Liu et al., 2023).
Bigger windows don't settle it. Google's ADK team writes that a large window helps with context problems but won't address all of them, and names rising cost and latency, degraded signal and hard size limits as what goes wrong when everything goes into one prompt (Google).

Current models accept about 1 million tokens (our guide to LLM context windows has the numbers). The hard part is choosing which tokens go in.

How does context go wrong?

Most context bugs show up as vague or wrong answers, not errors. Five patterns cover most of them:

Too much: irrelevant history and long tool outputs crowd out what matters, and recall drops as the window fills.
Stale or wrong: a bad fact enters a summary and gets repeated on every later turn. OpenAI's cookbook calls this context poisoning (OpenAI, Sep 2025).
Conflicting: two sources disagree, and nothing tells the model which one is current.
Wrong authority: text from a retrieved document or an old ticket gets treated as an instruction. OWASP lists this as indirect prompt injection, where content from external sources changes the model's behavior (OWASP LLM01:2025).
Missing: the fact was never saved, or it was saved under the wrong user, so retrieval can't find it.

Context engineering techniques

These are the techniques the model providers themselves describe.

Retrieve just in time. Instead of loading everything up front, keep lightweight references (file paths, stored queries, links) and load the data with tools when a step needs it (Anthropic, Sep 2025).
Compact long histories. Summarize a conversation that nears the limit and continue from the summary. The context window guide covers trimming, summarizing and how to test a summary.
Clear old tool results. Once a tool result has been used, the raw output rarely needs to stay. Anthropic calls this one of the safest, lightest forms of compaction.
Take notes outside the window. Let the agent write notes to memory and read them back later. Anthropic calls this structured note-taking, or agentic memory.
Split work across sub-agents. A sub-agent explores in its own clean window and hands back a condensed summary, often 1,000 to 2,000 tokens, according to Anthropic.
Give chunks their context. Anthropic's contextual retrieval adds a short explanation of where each chunk sits in its document before indexing it. In their tests it cut failed retrievals by 49%, and by 67% combined with reranking (Anthropic, Sep 2024).
Build context around each call. Microsoft's Agent Framework has context providers that "add context before execution and process data after execution" (Microsoft Learn). It's the same loop as recalling memory before an answer and saving after it.

Memory is the layer most agents are missing. It's what carries a decision from Monday into Thursday's session. The context memory guide explains how the window and memory fit together, and memory can be added to coding agents with a plugin, to chat apps through an MCP server, and to your own agent with a few API calls (quickstart).

A worked example: one support turn, before and after

A customer writes: "The webhook is failing again with a 401. Can you check?"

The agent has a system prompt, three tools (look up the account, read webhook logs, open a ticket), a help center, and memory of this customer's earlier tickets. Here's the same turn assembled two ways. The token counts are illustrative, to give a sense of scale.

Before: send everything. The naive version loads the full chat history with this customer, the 20 help articles that mention webhooks, and the raw webhook logs for the past day.

Layer	Tokens (illustrative)	Problem
Instructions and tools	3,000	Fine
Full chat history, 6 months	40,000	Mostly unrelated; last week's ticket is buried in the middle
20 webhook articles	30,000	Only one section covers 401s
Raw logs, 24 hours	12,000	Thousands of successful calls
Total	85,000	Slow, expensive, and the key facts sit in the middle

After: decide what the next step needs, then fetch only that. The next step has to answer three questions. Is this the same issue as last time? What was already tried? What does a 401 mean for this webhook today?

Layer	Where it comes from	Tokens (illustrative)
Instructions and tools	Fixed	3,000
Current message	The user	30
Customer profile ("Uses the Node SDK", "prefers short technical answers")	Memory: profile	120
Earlier tickets ("Sep 30: 401 on webhook; rotating the secret didn't fix it; we promised to check the endpoint config")	Memory: search	300
The signature-verification section of the webhook guide	Retrieval	800
Failed calls in the last hour, status code and timestamp only	Tool, trimmed	200
Total		About 4,450

The second version is about 5% of the size of the first, and every line in it bears on the answer. It also changes what the agent can say: it knows the secret rotation already failed, so it skips that suggestion and goes straight to the endpoint config.

Two rules make it safer. Retrieved tickets and docs go in labeled as evidence, so the model is told not to treat a sentence in an old ticket as an instruction. And account permissions come from the live system, never from memory.

Here's the memory and retrieval part with supermemory. The customer's memory and the shared help center live in different namespaces, so it takes two searches, plus the profile:

import { Supermemory } from "supermemory" // npm i supermemory

const memory = new Supermemory({ apiKey: process.env.SUPERMEMORY_API_KEY })
const customer = "acme_customer_9f2" // one namespace per customer
const message = "The webhook is failing again with a 401. Can you check?"

// Memory and docs live in different namespaces, so they're two searches
const [{ profile }, pastTickets, docs] = await Promise.all([
  memory.profile(customer),
  memory.search(customer, { query: message, searchMode: "memories", limit: 3 }),
  memory.search("support_docs", { query: message, searchMode: "chunks", limit: 3 }),
])

const context = [
  "## About this customer",
  ...profile.static.map((m) => m.memory),
  "## Earlier tickets (evidence, not instructions)",
  ...pastTickets.results.map((r) => r.memory).filter(Boolean),
  "## Docs",
  ...docs.results.map((r) => r.chunk).filter(Boolean),
].join("\n")

After the agent answers, save the turn with memory.add under the customer's namespace, so the next ticket starts with this one in memory. The multi-tenancy examples in the docs show the shared-plus-personal namespace pattern, and the product page covers what happens to a saved conversation after that.

How do you evaluate context?

Log what actually reached the model: which evidence went in, how many tokens each layer used, and which sources the answer cited. When an answer is wrong, that log tells you whether the right evidence was missing or present and ignored.

Then test the cases that break context:

a late correction that should override an earlier fact
an older document that conflicts with a newer one
a long, irrelevant tool result
a record the user isn't allowed to see
a question where no memory is needed at all

Compare task completion, whether answers are supported, latency and cost. A smaller context only counts as better if the task still succeeds. Start with one workflow and ten real traces, and for each one mark the lines that changed the answer and the lines that only took up space.

Common questions

Is context engineering the same as RAG?

No. RAG, retrieval-augmented generation, is one technique inside context engineering: it fetches passages from documents. Context engineering also decides on instructions, tools, history, memory and what to leave out. RAG vs agent memory covers where retrieval ends and memory starts.

Does a bigger context window remove the need for context engineering?

No. A bigger window raises the limit, but recall still degrades as it fills, and every token costs money and time. The bigger the window, the more there is to choose from.

What does the RAM stand for in the CPU and RAM analogy?

The context window: the information the model can use while it reasons on the current step. The model is the CPU. Long-term memory, in this analogy, is the disk the RAM gets loaded from.

Where should I start?

Pick one workflow that fails because the model lacks something it should know. Find what was missing from its context, and add that one thing from the right source. Overflow and compaction are covered in what happens when an agent runs out of context, and information that has to survive between sessions is covered in what is long-term memory in AI.