Why Did Your Agent Forget? A Memory Debugging Guide
Trace a memory failure through capture, processing, retrieval, context assembly, and answering before changing the architecture.
When an agent forgets a fact, trace the fact from its source to the final answer. It may never have been stored, may not yet be searchable, may be outside the current scope, may rank below the retrieval cutoff, or may have been ignored after retrieval. These failures need different fixes.
Start with one reproducible example. “Memory quality is bad” is too broad to debug. “A returning customer is asked to repeat a failed troubleshooting step recorded in ticket A” gives you a source, an expected behavior, and a query to inspect.
Why does an AI agent forget between sessions?
The new request may not contain the old messages, the application may have changed the user's memory scope, or stored content may not yet be searchable. First confirm that the relevant fact was written under the scope used by the new session. Then inspect retrieval and the actual model input. Increasing context size helps only when truncation is the stage losing the fact.
The AI SDK memory guide shows how to keep user scope separate from conversation identity. If the failure involves an outdated preference rather than a missing fact, inspect correction and retention behavior as well.
Capture a useful failure record
Record a trace ID, source record ID, event time, authorized scope, query, processing status, returned record IDs, context supplied to the model, and final answer. Include relevant model and retrieval settings.
Use a redacted or synthetic reproduction where possible. Full transcripts and raw prompts can contain private data, so logging everything indefinitely can create a second data-retention problem.
Find the first stage that lost the evidence
| Stage | Question | Typical next check |
|---|---|---|
| Capture | Did the application receive the fact? | Source event and request logs |
| Write | Was the intended content stored in the correct scope? | Write result, document ID, retry behavior |
| Processing | Was it available to search when the query ran? | Indexing state and ingestion lag |
| Retrieval | Did the query return the relevant record? | Scope, filters, ranking, query formulation |
| Context assembly | Was the evidence actually sent to the model? | Truncation, formatting, token budget |
| Answering | Did the model use the right evidence? | Conflicting records, instructions, question interpretation |
Stop at the first failed stage. Increasing the model's context window will not repair a write that went to the wrong user's scope.
Separate missing writes from delayed processing
A successful request can mean content was accepted without implying every derived memory is immediately available. Check the integration's documented processing behavior and the application's readiness handling.
For a read-after-write failure, compare an immediate query with a query after processing completes. If the later query succeeds, the fix may involve a readiness check or a temporary use of current-session context. Repeating the same write blindly can create duplicates without fixing the timing issue.
Check scope before tuning relevance
Compare the scope used for the original write with the scope used for retrieval. A new session ID used as the memory scope can accidentally prevent continuity. A scope accepted directly from an untrusted client can create an authorization flaw.
Verify document fetches, profile reads, tools, and cache lookups too. Search can be correctly scoped while another path is not. The tenant isolation guide provides concrete negative cases.
Inspect conflicting and stale records
Suppose the correct preference is “email updates for this case,” but retrieval returns an older global phone preference. Check whether the new statement was stored with its source, time, and narrower scope.
Do not resolve every conflict by choosing the newest timestamp. An imported historical record may have a recent ingestion time. A newer statement may apply to a different project. Preserve event time separately from ingestion time when the application needs that distinction.
Inspect what the model actually saw
The search response is not necessarily the model input. An application may trim it, summarize it, reorder it, or omit it when assembling a prompt.
Replay the case with the required evidence supplied explicitly. If the answer remains wrong, investigate reasoning and instructions. If it becomes correct, compare that input with the normal pipeline to find where useful evidence disappeared.
Keep retrieved content clearly identified as data. Old documents and conversations can contain instructions that should not override the application's rules or tool permissions.
Measure latency where it occurs
Measure search time, context preparation, time to first model output, and full task completion separately. Report tail latency as well as averages under representative concurrency.
Do not adopt a universal 200 ms or 300 ms budget from an unrelated article. A voice agent and an asynchronous research task have different tolerances. Establish the user-facing target first, then allocate time to each stage and test the whole path.
Turn the incident into a regression case
Save a minimal source history, query, expected behavior, and evidence requirement. Add a nearby negative case: a different customer, an old version of the fact, or a question with no supporting evidence.
Run the case before and after the fix. Then check whether the change increases irrelevant retrieval, context size, or latency elsewhere. The MemoryBench walkthrough can help structure broader evaluation, but the original incident should remain understandable on its own.
A useful postmortem states the first failed stage, the evidence for that conclusion, the change made, and the test that now catches it.
When the right evidence is retrieved but used poorly, inspect how the context is assembled.
If the failure follows a long session or compaction, use the context-limit troubleshooting guide.
Audit trajectories with a declared denominator
An output-only score can miss a tool retry, an unauthorized lookup that was later filtered, or a correct answer reached through unsupported reasoning. Inspect the sequence of events as well as the final response. Record run ID, step ID, source IDs, retrieval scope, processing state, selected context, tool outcome, and the expected next action. Redact raw customer text when IDs and status are sufficient.
Count failed runs separately from failed steps. In a hypothetical sample of 20 runs with five steps each, four failed steps across two runs mean a 4% step-failure rate and a 10% run-failure rate. Those denominators answer different questions. They are illustrative arithmetic, not findings from a 220-run study.
Label the first observed failure in each run and preserve evidence for the label. Have an ambiguous case marked unresolved rather than forcing it into a category. Compare the same replay cases after a fix, including retries and corrections. A claimed improvement needs the run set, labeling rules, versions, and result files that produced it.
For the ranking stage specifically, use the contextual reranking evaluation to separate missing candidates from poorly ordered evidence.
If you are evaluating a managed memory path, replay one failing case with Supermemory. Use a synthetic version of the source history and inspect ingestion, scope, and retrieved evidence at each step to find where its behavior differs from your current implementation.