Here is a strange fact about the language models underneath every AI agent: they have no memory at all. A model is stateless. It reads whatever is in front of it, produces an answer, and forgets the entire exchange the instant the session ends. Start a new conversation and it has never met you.
And yet the agents built on those models remember your name, recall your preferences, and pick up a task where they left off last week. That gap between a forgetful model and a remembering agent is bridged by a memory system, and the interesting part is that the design the field has converged on in 2026 looks almost exactly like the way human memory is described in cognitive science. Understanding it is the difference between an agent that feels like a goldfish and one that feels like a colleague.
Working memory: the now
Start with what the model can actually see: its context window. This is working memory, the short-term store, and it holds the present moment, the system prompt, the current conversation, and anything that was just retrieved from long-term memory. It is fast, it is where all the reasoning happens, and it is sharply limited. When the session ends, it is wiped.
The critical thing to grasp is that the agent can only reason over what is in working memory right now. Nothing in long-term storage helps unless it has been pulled into the context window first. This is the same attention budget that governs all model behaviour: the window is finite, so the entire art of agent memory is getting the right things into it, and keeping everything else out.
Long-term memory: what persists
Everything that survives beyond a single session lives in long-term memory, outside the context window, and gets retrieved only when relevant. Cognitive science splits this into three kinds, and AI agents have adopted the same three, because they turn out to map cleanly onto different jobs.
Semantic memory: what is true
Semantic memory is a store of facts, preferences, and domain knowledge, decoupled from when they were learned. "This customer is on the Pro plan." "Our support hours are 9 to 5." "The user prefers concise answers." It does not matter when the agent learned these; they are simply true and available. This is the memory that makes an agent feel like it knows you and your world.
Episodic memory: what happened
Episodic memory logs specific past events, full conversations, and completed task runs, tied to a point in time. "Last week we discussed the renewal." "On Tuesday the agent ran the export and it failed." Where semantic memory holds standing facts, episodic memory holds the timeline of what actually occurred, which lets an agent reference history, learn from past runs, and avoid repeating itself.
Procedural memory: how to act
Procedural memory is the agent's knowledge of how to do things: skills, tool-usage patterns, workflows, and the behavioural rules it follows. It is the difference between an agent that has to be told the steps every time and one that has internalised "this is how we process a refund here." If skills are how you hand an agent a procedure, procedural memory is where that know-how lives.
| Memory type | Holds | Example |
|---|---|---|
| Working | The current context | This conversation, right now |
| Semantic | Facts and preferences | "This customer is on the Pro plan" |
| Episodic | Timestamped events | "Last week we discussed the renewal" |
| Procedural | How to do things | "This is how we process a refund" |
Retrieval: the part that actually matters
Having memories is only half the system. The hard, valuable part is retrieval: deciding which memories to pull from long-term storage into working memory for the task at hand. Get this right and the agent has exactly what it needs. Get it wrong and it either misses crucial context or drowns the model in irrelevant history.
This is also where the economics live. A naive approach, cramming all your history into every prompt, is slow, expensive, and actually degrades quality as the window fills. A real memory layer stores memories outside the window and retrieves only the relevant slice on demand, often using a hybrid of vector search and graph traversal to find what matters. The payoff is large: Mem0's 2026 research reported roughly 90% token-cost savings and dramatically lower latency versus naive context stuffing. Systems like Letta take a similar line, keeping full conversation history out of context by default and recalling it only when needed.
Less, but exactly the right less. It is the same principle as context engineering, applied to memory.
Why this matters if you're building agents
If you are building anything beyond a single-turn chatbot, memory is not optional, it is the thing that makes an agent useful over time. A few practical takeaways:
- Match the memory type to the need. Storing standing facts? That is semantic. Need the agent to recall a past interaction? Episodic. Teaching it a repeatable procedure? Procedural. Reaching for the wrong one is a common early mistake.
- Invest in retrieval, not just storage. The quality of an agent's memory is mostly the quality of what it retrieves. This is the same discipline as advanced RAG, because it largely is retrieval.
- Keep working memory lean. Do not stuff everything in on principle. Pull in the relevant memories, leave the rest in storage, and your agent will be faster, cheaper, and sharper.
- You usually don't build this from scratch. Memory frameworks now handle the storage and retrieval plumbing, so you can focus on what your agent should remember rather than how to persist it.
The shift worth internalising is that a capable agent is not just a clever model. It is a clever model wired to a well-designed memory. The model provides the reasoning; the memory provides the continuity that turns a series of forgettable exchanges into something that actually accumulates knowledge about you and your work.
Designing agents that remember well, and evaluating whether they do, is part of what we build. If you are moving past chatbots toward agents that need to persist and personalise, get in touch.
Sources
Frequently asked questions
- How does an AI agent remember things?
- A language model itself is stateless and forgets everything when a session ends. Agents add a memory system on top: working memory holds the current context, and long-term stores hold the rest across sessions. When something is relevant, the agent retrieves it from long-term memory and places it into the context window so the model can use it.
- What are the types of AI agent memory?
- The main types mirror human cognition. Working memory is short-term, held in the context window right now. Long-term memory splits into semantic memory (facts and preferences), episodic memory (specific past events and conversations, timestamped), and procedural memory (how to do things, like skills and workflows).
- What is the difference between episodic and semantic memory in AI agents?
- Semantic memory stores facts decoupled from when they were learned, such as 'this customer is on the Pro plan.' Episodic memory stores specific events tied to a time, such as 'last week we discussed the renewal.' Semantic is what is true; episodic is what happened.
- Why does AI agent memory matter for cost and performance?
- Because a model can only reason over what is in its context window, and stuffing everything in is slow and expensive. A good memory layer retrieves only the relevant memories on demand, which keeps the context lean. Mem0's research reported around 90% token-cost savings and much lower latency versus naive context stuffing.