Long-term memory in AI: what it is and how it works
Long-term memory in AI is information an application saves outside the model and brings back in later conversations, so an assistant can remember users, decisions and unfinished work across sessions. It lives in files, databases or a memory service, and using it doesn't change the model's weights.
It has to live outside the model because the model keeps nothing between requests. Anthropic's Messages API is stateless, so each call only knows what's sent with it, and a bigger context window holds more of one session but nothing of the next. That gap is measurable: LongMemEval, a benchmark of 500 questions about long chat histories, found commercial chat assistants and long-context models showed a 30% accuracy drop when they had to remember information across sustained interactions (Wu et al., 2024). If you're new to the topic, how AI memory actually works is a beginner's guide.
A useful memory feature does more than save a transcript. It lets a returning user continue work, explains where remembered details came from, and responds correctly when those details change.
What persists, and what does the model actually see?
Think of a project assistant with three separate records: the conversation history, a decision log, and the current project settings. The history can reconstruct what people said. The decision log can explain why an approach was chosen. The settings tell the application which configuration is active now.
The model sees only the information provided to a particular call, whether that comes from the application, a platform's managed conversation state, or a retrieval tool. Keeping data somewhere does not guarantee that the relevant part reaches the next answer.
| Mechanism | Useful for | Does not establish by itself |
|---|---|---|
| Conversation history | Reopening a discussion | Relevant recall across unrelated sessions |
| A saved profile | Reusing selected preferences | A complete account of every interaction |
| Document retrieval | Answering from a stored corpus | Which user-specific decision remains current |
| A decision record | Preserving rationale and revisions | Current permission to perform an action |
| Model fine-tuning | Changing learned behavior | Immediate, exact, user-controlled factual updates |
Products can combine these mechanisms. Check which records your application retains and which of them it supplies to a new session.
Why does a chatbot forget a previous decision?
The failure can happen at several different points. The decision may never have been saved. It may have been saved under the wrong user or project. Retrieval may return an older version. Or the correct record may arrive in the prompt but get lost among irrelevant material.
Suppose a team selects PostgreSQL after evaluating another database. Three days later, an assistant recommends the rejected option. Before adding a new memory provider, inspect four things:
- Is the decision stored with its project and source?
- Does a fresh session use the same authorized project identity?
- Does retrieval return the decision and its rationale?
- Does the answer recognize it as the active decision rather than an unresolved suggestion?
Each failure implies a different fix. The memory debugging guide walks through that trace.
What should an AI application remember?
Prefer information that helps future tasks: explicit preferences, accepted decisions, unfinished work, and source references worth revisiting. Keep the original scope. “Use short answers for status updates” should not prevent a detailed architecture explanation when the user requests one.
Avoid treating all generated text as user truth. A model's suggested plan is not an accepted plan, and a draft message is not proof that the message was sent. Preserve the difference between proposed, approved, and completed work.
For a decision, store the choice, rationale, alternatives rejected, relevant date, project, and source. For a preference, store what applies, to whom, and any conditions. For an unfinished task, store the last confirmed state and the next action rather than a speculative completion.
How should corrections work?
When someone changes a preference, current answers should use the new value. When someone asks what was true last month, the application may need the older value too. Decide whether that historical question is part of the product before choosing how to retain revisions.
Do not rely on “newest text wins” for every case. A document uploaded today can describe an older event. A casual suggestion can be newer than an approved policy without replacing it. Use source authority, scope, and effective time where the workload requires them.
Forgetting, expiration, and deletion also serve different purposes. Excluding an obsolete note from routine answers does not necessarily remove it from storage. Use the memory lifecycle guide to define the behavior people can expect.
Does long-term memory require a vector database?
No. A small set of preferences can be retrieved by user ID from an ordinary database. Versioned project notes can be searched as files. Semantic search becomes useful when there are many records and users describe the same idea in different words.
Likewise, a graph is an option for explicit relationships, not a requirement for remembering a preference. Choose storage and retrieval around the questions you need to answer. The memory architecture guide explains those choices without assuming a fixed number of services.
How do you test whether memory is working?
Use an example the model could not plausibly know in advance, such as a fictional project's chosen code name. Save it, start a fresh session, and ask a related question. Then inspect the retrieved evidence as well as the answer.
Repeat with another user, a correction, a deleted record, and a question unrelated to the memory. Good memory should retrieve the right information when relevant and stay out of the way when it is not. A system that injects every saved fact into every answer has persistence but poor selection.
For supermemory's approach to selected user context, see the profile documentation, and the quickstart for the save-and-recall loop in code. The product page explains how saved conversations become memories that get updated and forgotten. To implement continuity in an application, start with the AI SDK integration, compare the three ways to add long-term memory, or compare memory API options.
The first success criterion is a completed user task: the assistant resumes work using the right earlier decision, without requiring the user to reconstruct the entire conversation.
For an application-level trial, get started with Supermemory and implement one preference that should survive a new conversation. Run the correction and deletion checks too, so the first demonstration covers control as well as recall.