When an AI assistant seems to remember you, the language model itself may not have learned anything new. A common way to implement AI memory is as a feature around the model: software keeps a fact, a past message, a summary or a file, then supplies relevant pieces again in a later request. The model uses that material as context—information available for the answer it is producing now.
This distinction matters because three different things are often called memory. Model weights are the numerical settings changed during training. Context is the material supplied for one model call. Product memory is information saved by the application outside the model and brought back when it may be useful.
The three routes look similar from the chat box, but only one changes the model. In the diagram, notice that a prompt and a stored preference both flow into context; training follows a separate path to new weights.
How it works in outline
During an ordinary model call, the application sends a sequence of tokens and the model generates an answer using frozen weights. To keep a conversation going, the application must preserve state: it can send earlier messages again or refer to a stored conversation. OpenAI’s API documentation makes the boundary explicit—individual text-generation requests are independent and stateless unless conversation state is supplied or persisted by the surrounding service.
The phrase in-context learning can make this sound more permanent than it is. In the GPT-3 paper, examples placed in the prompt changed how the model performed a task without gradient updates or fine-tuning. “Learning” there describes adaptation within the supplied context, not a rewrite of the parameters.
Product memory adds a storage-and-retrieval step. The product decides what to keep, how to summarise it and when to insert it into a later context. The original retrieval-augmented generation research called the model’s weights parametric memory and an external document index non-parametric memory. Consumer memory features use different implementations, but the useful distinction remains: stored information can be updated or deleted without retraining the language model.
Try the vegetarian test
Suppose you tell an assistant, “Remember that I’m vegetarian.” Four different things can happen:
- In this conversation: the sentence is still in the context, so the next answer can use it.
- In a later conversation: a memory-enabled product can save the preference and supply it again.
- With no saved state: a fresh, stateless request will not contain the preference unless you send it again.
- During training: selected data are processed in a separate optimisation run that changes weights. Your sentence did not do that merely by appearing in a prompt.
A provider may separately use eligible conversations in a later model-improvement process, depending on the product, account and data controls. That is another reason not to merge “remembering me” with “training on my chat.” In ChatGPT, for example, memory and model-improvement controls are separate, and its current documentation says saved memories can be stored separately from chat history.
Where you meet it
You meet this distinction in chat history, saved preferences, custom instructions, project memory, customer profiles, document retrieval and agents that write progress to files. If you build with an API, conversation state and long-term memory are parts of your application architecture, not abilities you get automatically from a bare model call.
Before trusting a memory feature with something important, ask four questions: What is saved? Where can I inspect or correct it? Does deleting the chat also delete the saved memory? Are my conversations eligible for later model improvement? The answers are product-specific and can change. The durable mental model is simpler: an assistant may remember you without the model learning you.