Why a Chatbot Remembers a Preference but Still Gets It Wrong
A chatbot can retrieve a user's preference and still apply it incorrectly. The record may be outdated, inferred from a one-time request, or valid only for a particular project. Diagnose those cases before collecting more personal information.
The goal is not maximum recall of every preference-like sentence. It is the correct preference, under the right conditions, with a correction path the user understands.
Distinguish explicit settings from inferred patterns
“I prefer metric units” is an explicit preference. Asking one question about kilometers may not be. Preserve whether a fact was directly stated, inferred, or confirmed later. An inference should not quietly override a maintained setting.
For a high-impact choice such as a communication channel or account detail, consult the authoritative application record. Memory can explain previous context, but it should not bypass current permissions or transactional state.
Resolve the scope of an exception
“Give me more detail for this incident” can be a local exception to a general preference for short answers. Save the exception with its task scope rather than replacing the lasting preference for every future conversation.
A practical fixture has three records: a general preference, a project-specific preference, and a current request. Define the expected answer for each combination. That makes precedence a testable product rule rather than a prompt that merely asks the model to use good judgment.
Separate retrieval from application
Log the preference IDs and versions supplied to the answer step. If the new preference is missing, inspect capture, processing, scope, and ranking. If it is present but ignored, inspect conflicting instructions and context assembly.
Do not judge success only by asking “What do you remember about me?” A chatbot may list a preference correctly but fail to apply it while completing a real task. Ask it to write the report, choose the format, or continue the workflow that the preference should influence.
Verify the correction in a new session
Use a fictional preference, update it explicitly, and start another conversation. Repeat under a different user and a different project. Then remove the record and check that a source resync does not restore it unexpectedly.
Report failures by cause rather than one broad personalization score. An irrelevant formatting choice and another user's preference appearing in an answer need different responses.
The preference implementation guide covers persistence and scope. To try the behavior end to end, start with Supermemory and one preference that your test user can inspect, change, and remove.