Engineering Persistent Memory for AI Agents
How we approach the engineering challenges of building memory systems that remain accurate, contextual and reliable over time.
The difficult part isn't remembering. It's knowing what to remember.
An AI agent can retain every conversation it has ever had with a customer and still have a fundamentally flawed understanding of that customer.
It might remember a budget that has since changed, mistake a tentative intention for a confirmed decision, or turn an inference into something the customer supposedly said. It might retrieve information that is technically relevant but no longer true. Over time, these small inaccuracies compound, gradually shaping an understanding of someone that no longer reflects reality.
This is the problem with treating memory as a storage or retrieval problem alone.
Persistent memory is already a well-established direction in AI engineering. There are increasingly sophisticated ways to retain conversational history, extract facts, generate embeddings and retrieve relevant context. But making information available to an agent is not the same as giving it a reliable understanding of the world.
A useful memory system needs to do considerably more. It must distinguish observations from assumptions, understand how facts relate to one another, recognize when something has changed, preserve the history behind those changes and establish clear rules about what an agent can treat as true.
It also needs to do all of this without introducing meaningful latency, compromising data isolation or making the underlying system unnecessarily difficult to maintain.
These are the problems we've been working through at Meridian as we integrate persistent memory into the AI systems we build.
Our approach combines Supermemory's fact extraction and retrieval capabilities with an application-controlled layer for structured records, profile versioning and human-approved changes. The result is an architecture designed not simply to help agents recall previous interactions, but to maintain a consistent and evolving understanding of the people and businesses they work with.
In this article, we unpack the architecture, the engineering decisions behind it, what we learned from testing it, and how the same memory can support agents operating across chat, voice and human workflows.
01Memory is more than storing information
A common misconception is that giving an AI agent memory simply means giving it access to previous conversations. That is a starting point, but it isn't enough.
Consider a customer who tells an agent in January that they're working with Supplier A. In March, they mention that they've moved to Supplier B. In June, they explain that the decision was driven by delivery delays. A system that simply retrieves old conversations might surface any of these statements. But a system with meaningful memory should understand how they relate to one another.
It should know that Supplier B is the current supplier, that Supplier A was previously used, and that the change happened because of delivery issues. More importantly, it should distinguish between something the customer explicitly said and something the system inferred from the conversation.
We think about this problem through three complementary layers of memory.
Short-term memory: what's happening now. This is the immediate context of an active conversation. It contains the messages, information and decisions needed to keep the current interaction coherent. We store it in a conventional database alongside the conversation.
Episodic memory: what happened before. Every interaction produces an immutable record of what took place. These records preserve the history of conversations, decisions and events, giving the agent a reliable account of previous interactions.
Semantic memory: what we know today. This is the evolving understanding of a person, customer or case. It brings together information gathered across multiple interactions into a versioned profile that can be updated as circumstances change.
The distinction is important. Episodic memory tells the agent what happened. Semantic memory helps it understand what is currently true. The difficult part is keeping the two connected without allowing the system to confuse observations, assumptions and outdated information.
There are three challenges that every persistent memory system needs to address.
What was said versus what was inferred. A prospect might say, "We could look at this next quarter, depending on the pricing." That is not the same as saying they intend to purchase next quarter. Yet a language model summarizing the conversation might turn that conditional statement into a definitive intention.
If the agent later treats that interpretation as something the prospect explicitly confirmed, the system has effectively put words in their mouth.
What was true then versus what is true now. People change their minds. Budgets change, priorities shift and circumstances evolve. A memory system must preserve the history of those changes while making it clear which information is current. Otherwise, an agent risks acting on outdated information or losing the context behind an important decision.
What the system knows versus what it is allowed to believe. Perhaps the most important question is who gets to decide what becomes part of a person's lasting profile. An AI model can propose that something has changed, but that doesn't necessarily mean it should be allowed to rewrite the record automatically. Without clear rules governing how information enters and changes memory, a system's understanding of someone can gradually drift away from reality.
Two additional constraints influence how we implement all of this. First, memory must be fast. An agent shouldn't make someone wait while it searches through previous interactions. Second, information belonging to one person must never appear in another person's conversation. Privacy and isolation must be enforced by the architecture, not left to the model to interpret.
02Choosing the foundation for our memory architecture
We use Supermemory as the underlying memory infrastructure for our agents. The decision came down to a practical consideration: we wanted to benefit from the complexity of a dedicated memory engine without surrendering control over how our agents store, interpret and use information. There are two distinct ways we use the platform.
An engine that understands what people say. Supermemory processes conversation transcripts and identifies individual facts worth retaining. We provide instructions specific to each product to guide what it should extract.
The resulting memories can be searched using similarity thresholds and metadata filters. The system also identifies information that has been inferred rather than explicitly stated, as well as facts that have subsequently been replaced. Each fact is associated with the date of the interaction from which it originated. These capabilities address several of the challenges above: distinguishing observations from inferences, recognizing changes over time, and preserving the context behind a piece of information.
A store for the records we write ourselves. We also use Supermemory to store structured records that we create and manage ourselves. Each person, customer or case gets a private namespace. Within it, we store records such as session summaries, proposed changes and profile versions, exactly as we wrote them, under identifiers that we control.
This distinction is important. We want the flexibility of an external memory engine, but we also want control over the records our agents rely on.
Why not build everything ourselves?
A memory system involves considerably more than a vector database. It requires document processing, extraction, fact relationships, indexing, retrieval and ongoing infrastructure management. Supermemory provides much of that underlying functionality, allowing us to concentrate our engineering effort on the parts that directly affect our products: what gets remembered, how information is updated, who can approve changes and when an agent is allowed to use that information.
Memory should improve an agent, not become a prerequisite for it to function. If the memory service is unavailable or disabled, the agent should continue operating using its existing conversational context and capabilities. In our implementation, memory can be enabled or disabled independently through configuration, allowing us to roll it out gradually, isolate issues and disable it without disrupting the underlying agent.
One qualification is worth making: we haven't conducted a formal benchmark comparing Supermemory against a self-hosted implementation of the same architecture. Our decision is based on the capabilities we've tested, the behavior we've observed and the degree of control we retain through our own application logic.
03How the memory system works
Every document we send to Supermemory follows one of two processing paths. Understanding the difference between them is central to understanding how the system works.
Path one: making information searchable. The first path is conventional document retrieval. When a document is submitted, it enters a processing queue. Its contents are extracted, which can include transcription or fetching the contents of a web page when necessary. The resulting text is divided into smaller chunks, converted into embeddings and indexed for search. Once processing is complete, those chunks become searchable.
We don't need to manage the chunking strategy or embedding model ourselves. More importantly, structured records that we author are kept on this path exactly as written. They aren't rewritten or interpreted by the extraction engine. This path is useful when the agent needs to retrieve information from an existing document or record.
Path two: extracting facts and building memory. The second path is what Supermemory calls dreaming. Rather than simply making a document searchable, this process uses a model to interpret its contents and extract individual facts. Those facts are then connected to existing memories, forming a graph of related information. The system also maintains a profile containing information about the user, separating longer-lasting facts from more recent context.
We deliberately send only raw conversation transcripts through this process. We don't use Supermemory's generated profile as the source of truth for our agents. The profiles they rely on are assembled and updated through rules that we control. This allows us to benefit from automated fact extraction without handing over authority over the final record.
Making extraction available when the agent needs it. Dreaming runs in a batched mode by default, meaning fact extraction can finish some time after the initial document processing has completed. In our tests, no extracted memories had appeared after seven minutes. That delay was too long for our intended use case.
We therefore submit transcripts using the instant setting. This adds one operation per document, but allows us to make extracted memories available much sooner. In our subsequent tests, facts appeared between 22 and 60 seconds after upload.
The distinction between the two paths is fundamental: one makes information searchable; the other attempts to understand what that information means and how it relates to what the system already knows.
A graph of facts that understands change
Extracting facts is only part of the problem. The system also needs to understand how new information relates to existing information. Supermemory represents these relationships through a graph of facts, and three relationships are particularly important to our implementation. The examples here are illustrative.
Updates: when something changes. Suppose a customer previously worked with Supplier A but later moves to Supplier B. The new fact replaces the old one as the current version of that information. The previous fact remains in the record, marked as no longer current. The system can therefore answer two different questions: who is the customer working with today, and who were they working with previously?
Extensions: when new information adds context. Not every new fact contradicts an existing one. A customer might initially say they prefer short meetings. In a later conversation, they explain that this is because their mornings are fully booked. The second fact adds context to the first. Both remain valid, and together they provide a more complete understanding of the customer.
Derivations: when information is inferred. Sometimes a useful conclusion emerges from several pieces of information, even though nobody explicitly stated it. A customer might repeatedly mention that their mornings are occupied and that they prefer meetings later in the day. A system might infer that afternoon meetings are generally more convenient. That conclusion can be useful, but it is still an inference, not something the customer explicitly confirmed.
We preserve that distinction. Our retrieval process excludes derived facts and facts that have been replaced. Agents therefore work from information that was explicitly stated and remains current, rather than treating a model's interpretation as an established fact.
Inference can still be valuable for internal analysis. But we don't want an agent presenting an assumption as something a person actually said.
04Retrieving the right information, safely
A memory system is only useful if it can retrieve the right information at the right time. Supermemory offers three retrieval modes:
- Hybrid: searches across both extracted memories and document chunks.
- Memories: searches extracted facts.
- Chunks: searches the original indexed content.
For our agents, we use the memories-only mode. This is intentional. We want the agent to retrieve concise, structured information about a person or case, rather than returning fragments of previous conversations that may be outdated or difficult to interpret.
We also adjusted the similarity threshold from the default of 0.3 to 0.55. In our testing, even loosely related facts could receive similarity scores between 0.5 and 0.7. One short, vague question matched an unrelated fact at 0.57. A higher threshold helps reduce these weak matches, although it doesn't eliminate them. We therefore combine it with a hard limit of three retrieved facts.
Supermemory also supports reranking and query rewriting. Both introduce additional processing and latency. Given that our agents have a strict one-second recall budget, we chose not to use either.
Forgetting matters as much as remembering. People should be able to change what a system knows about them. Supermemory supports soft expiration and forgetting, allowing facts to be removed from active search without physically erasing the underlying records. This provides a mechanism for controlling what remains available to the agent while retaining the underlying information where appropriate.
Isolation must be enforced structurally. Each person or case gets its own namespace, backed by a separate vector index. This means that one person's memories cannot appear in another person's search results simply because the two happen to discuss similar subjects.
Metadata filters, such as record type or interaction number, operate within that boundary. They provide additional organization, but they aren't the mechanism responsible for separating one person's information from another's. The principle is straightforward: an agent should never be able to retrieve information that it was never authorized to access.
05Eight engineering principles we've learned
The architecture described above is the result of implementing persistent memory in a real product and working through the problems that emerge along the way. These are eight principles we now apply to every memory system we build.
The first is the one that matters most, so we start with how a conversation becomes part of the record.
Let models propose changes. Let people confirm them.
A model can summarize an interaction and identify information that may need to be added or changed. But we don't let it silently rewrite a person's profile. Instead, the model produces a proposed update. Our application code assembles the profile version by version, and each version remains immutable once written. A proposed change is applied only after the person it describes has confirmed it or an authorized team member has approved it. This creates a clear separation between what a model interprets and what the system officially records.
Keep your own records exactly as written.
We send raw transcripts to Supermemory for fact extraction. Our own records, including session summaries, proposed changes and profile versions, are stored as structured JSON. They are not passed through the extraction process or rewritten by another model. This ensures that the system's original records remain distinct from the facts generated by the memory engine.
Control your own record identifiers.
Every record we create has an identifier that we generate ourselves. This becomes important when background jobs fail or need to be retried. Before writing a record, a job can check whether its identifier already exists and avoid creating a duplicate. Our background jobs currently use three attempts. If a process restarts or an operation fails, predictable identifiers make retries safer and easier to manage.
Memory should never interrupt the conversation.
We don't want an agent to have to ask another model for permission to remember something before it can respond. That would introduce an additional model round trip before the agent's first word. Instead, memory retrieval runs in parallel with the agent's other retrieval operations. We give it a maximum of one second and allow it to fail silently if it takes longer. The agent continues working with the context it already has rather than making the person wait.
Present memories as notes, never as direct quotations.
An agent should understand the difference between something a person explicitly said and a summary of information extracted from previous interactions. We therefore introduce retrieved memories into the agent's context as dated notes, not as quotations. The agent is instructed to use those notes naturally, without reciting them as though they were the person's exact words. Quotation marks are reserved for text the person actually said.
Derive identity from the system, not the conversation.
An agent must know whose memory it is accessing, but that identity should never come from something a person says during a conversation. We derive the namespace from the identity supplied by our platform when a session begins. This prevents conversational content from influencing which person's memory the agent accesses. Identity and access boundaries belong to the application architecture, not the language model.
Memory must be optional and fast.
A memory system should make an agent better, not make it slower or less reliable. We set two performance limits: two seconds at the start of a session and one second per subsequent turn. If memory is unavailable, disabled or too slow, the agent continues without it. This is particularly important in voice applications, where even small delays are noticeable. The conversation should never depend on a memory service responding perfectly every time.
Be deliberate about what enters memory and what appears in logs.
Not every piece of information should become part of a person's lasting record. Content flagged by our safety checks never reaches the transcript or subsequent summaries. We also write and test our own extraction instructions for each product. Without sufficient guidance, we found that the extraction engine could invent facts. In one test, an agent's own introduction became a stored fact suggesting that the agent itself was a person. In another, the system inferred a pronoun from a first name. These examples reinforced the importance of treating extraction as something that needs to be tested, not simply enabled. We apply the same discipline to logging. Our logs capture error types and record identifiers, rather than potentially sensitive message content, because an error message can contain information a person typed.
06One memory across chat, voice and human teams
The architecture becomes easier to understand when we consider a practical application. Imagine an enterprise sales operation in which a conversational agent handles inbound enquiries, an outbound voice agent follows up with prospects, and a human sales team manages the opportunities.
A prospect contacts the business on WhatsApp on Monday, explains their requirements and discusses a potential budget. On Thursday, a voice agent calls them. Without shared memory, the agent starts from scratch. It asks the same questions, repeats information the prospect has already provided and risks creating an unnecessarily frustrating experience. Each agent may be working correctly in isolation, but the overall experience breaks down because none of them understands what happened before.
A shared memory system changes this.
One memory for every channel. Each lead has a private namespace shared by the chat agent, voice agent and relevant human workflows. When the voice agent prepares to call, it can retrieve the information gathered during the earlier WhatsApp conversation. The namespace is derived from the lead's identity in our platform, not from anything said during the call.
Remember what was actually said. Suppose the prospect wrote, "Maybe next quarter, if the pricing works." The system should preserve the condition attached to that statement. It shouldn't turn it into a confirmed intention to purchase. When the voice agent calls, it can acknowledge that the prospect was considering a purchase next quarter, subject to pricing, without assuming that a decision has been made.
Recognize when circumstances change. Budgets and timelines often change during a sales process. If a prospect initially discusses a budget for one quarter and later revises it, the memory graph maintains the relationship between the two facts. The latest information becomes the current version, while the previous information remains available as historical context. The voice agent therefore has a way to distinguish the prospect's current position from what they said a month earlier.
Keep people involved in important decisions. Models can identify possible changes to a lead's profile, but those changes don't automatically become established facts. A prospect can confirm an update during a conversation, or a team member can approve it. Staff can also inspect the proposed changes and the underlying records. This gives the human team visibility into what the system remembers and why.
Make memory work within the constraints of voice. Voice agents have particularly little room for unnecessary latency. We load the relevant context before the call, run recall in parallel and enforce the same time limits described earlier. If memory is slow or unavailable, the agent proceeds without it.
The result is one continuous understanding of a lead, regardless of whether the next interaction happens through WhatsApp, a phone call or a human sales representative. The same architecture extends beyond sales. Replace the lead with a customer account, supplier or case file, and the underlying principles remain the same.
07Testing the system: what we learned from implementation
The architecture described here is based on our actual implementation and testing, not just the capabilities described in product documentation. In late October 2026, we ran a series of tests against the Supermemory API to understand how the system behaved under our intended operating conditions. Four findings had the greatest influence on our configuration.
| What we observed | What we changed |
|---|---|
| With instant dreaming, facts appeared 22–60 seconds after upload. With the default mode, none appeared within seven minutes. | We submit transcripts using instant dreaming. |
| Typical recall searches took approximately 0.5 seconds, but one search took 38 seconds. | We enforce a one-second recall limit and continue without memory if it is exceeded. |
| Similarity scores ranged from 0.5 to 0.7, including weak matches. A vague question matched an unrelated fact at 0.57. | We use a 0.55 threshold and retrieve no more than three facts. |
| Unit tests against a simulated memory service provided fast feedback but couldn't reproduce real API behavior. | We maintain 27 unit tests and a separate end-to-end smoke test against the real API. |
The similarity threshold deserves particular attention. A fixed threshold cannot reliably distinguish relevant information from irrelevant information in every situation. A score of 0.57, for example, might represent a useful match in one context and an unrelated result in another. This is why we combine the threshold with a strict limit on the number of facts retrieved. We would rather give the agent a small amount of relevant context than overwhelm it with information that might not apply.
We also found that testing against a simulated service was not enough. Our 27 unit tests help us verify application logic quickly and consistently, but they cannot tell us how the actual memory service behaves. We therefore added a smoke test that exercises the complete integration against the real API, using a fictional person and taking approximately two minutes to run.
A full real-service session also allowed us to generate an initial profile and identify a bug that our simulated tests hadn't caught. That issue has since been fixed and covered by a test.
The broader lesson is that memory systems need to be tested at two levels: whether our own logic behaves as expected, and whether the external infrastructure behaves as we need it to. Both matter.
08Where this pattern applies next
Nothing in this design depends on the sector. Any agent that meets the same person, customer or case more than once has the same needs: a record of what happened, a profile that changes only through defined rules, recall that is fast and skippable, and a way to recognize change.
Customer service agents, account management agents, onboarding agents and internal operations agents all fit that description. We are applying the same memory design to the agents we build across our platform.
There is still work ahead. Background jobs currently run as in-process tasks with three attempts and will move to a durable queue as we scale beyond one server. Human reviewers will get a dedicated view of what the system remembers and what it proposes. Recording a person's confirmation in a later conversation also needs its own step in the conversation plan, and each type of conversation needs extraction guidance of its own.
These are implementation challenges, but they point towards something much bigger. Imagine an agent that remembers why a customer first approached a business three years ago. That recognizes their priorities have changed since then. That understands the history behind a decision, rather than simply retrieving the last conversation. And that can bring this context into a new interaction without requiring the person to explain themselves all over again.
Not because it remembers everything, but because it knows what matters, understands what has changed, and knows when to ask.
That is the kind of continuity we believe the next generation of AI systems should deliver.
Start a conversation.
- Middle East
- Europe
- LatAm