SUNDAY NEWSLETTER #004 · PUBLISHED
Build stickier AI products with memory
Dhruv Dang
Founder & Principal Engineer, Steadily
This Sunday's newsletter is about adding memory and personalization to AI products. AI has disrupted many moats that used to protect software, but memory is proving itself to be a strong new one. How do we build products that learn from each user, so users never have to repeat themselves?
Key takeaways:
- Memory is a moat. A product that has learned how each user works is hard to copy and hard to leave.
- Look for repetition. Anywhere users repeat themselves is an opportunity for memory.
- Five primitives. Any memory system can be built on five tools: remember, recall, search, replace and forget.
- Quality comes down to two things. How well the system decides what to remember, and how well it recalls the right memories at the right time.
Memory and personalization is the new moat
AI has made software cheap to build. A feature that took a team a quarter can now be cloned in a weekend, so features alone no longer protect a product.
Memory is different. The more time a user spends with a product that has memory, the better it gets at serving them. A competitor can copy the features, but they start from zero on what the product has learned about each user. Switching means teaching a new product everything all over again.
That's why I think it's now essential for SaaS and AI product companies to implement some form of memory and personalization.
What is AI memory?
Memory is how AI-powered applications remember what users want. With memory, an app can:
- Do what users ask with more accuracy.
- Anticipate what users need before they ask.
- Adapt as users' needs change over time.
How does memory work?
Language models are stateless. This means every request to a model starts from a blank slate, and it only knows what we put in its context window. Memory fills that gap: an app can save facts and preferences about each user, then bring the relevant ones back into context when they're needed.
What an app should remember about its users depends on the problem it is solving. For example:
- An AI writing app learns a user's writing style, so it can draft documents in the user's voice.
- AI billing software learns how its users' customers accrue charges and are invoiced, so users don't have to manually edit each customer's generated invoices.
- An AI recipe app learns a user's dietary restrictions, so it suggests recipes and meal plans that match their health requirements.
Memory isn't the only way to personalize
Two other approaches are worth knowing:
- Custom user prompts. Instructions a user writes once (writing style, dos and don'ts) that get appended to your app's internal prompt(s). You can offer this to users as a text field in your app's UI.
- Categorization. Routing users into different experiences, prompts or agents based on their profile. For example, admin users' prompts go through a different AI pipeline from regular users' prompts.
What are the advantages of memory?
The most obvious advantage is that users no longer have to do the same mundane, repetitive tasks again and again. For example, a writing app that remembers how you sign off your emails adds it to every draft, so you never have to type it again. That's why repetitive user behaviour is a strong signal of where to build memory into your product.
Memory also lets a product adapt without being told. Nobody has to fill in a settings page: a user's real interactions steer the product toward what they want, organically and over time.
Upgrade your internal tools with memory
Memory isn't just for customer-facing products. Internal tools are a great place to start, because they're often used by a small, consistent group of people doing specific, repetitive tasks. For example, an internal agent that writes monthly status reports can remember leadership's changing requirements for what those reports should contain.
A pathway to discover new features
Memories are also a great way to source next-generation product features. If many users end up with the same memory, it should probably become a universal feature or default. Your users' memories are effectively a running list of what they wish your product already did.
My approach to building memory: five primitives
While every product's memories are different, the underlying machinery is the same. I build every memory system on five memory primitives: remember, recall, search, replace and forget. In an agentic product, each primitive is simply a tool the agent can call.
remember: save a new memory.recall: load specific memories into context.search: find memories that match a query.replace: swap an old memory for a new one.forget: delete a memory.
Because these are tools, we can encode rules on the conditions that permit each one to be called. "Only forget a memory when the user explicitly asks" stops an agent from deleting important memories without the user's permission.
I also enforce an important constraint on memories themselves: one memory always maps to one fact. For example, "User lives in Vancouver, BC, Canada" and "User prefers metric measurements" are two distinct memories. Single-fact memories are much easier to maintain and replace over time.
A note on memory replacement
Memories often go stale. Facts change (a user moves from Toronto to Vancouver), and sometimes a memory was based on incorrect information to begin with. A system that can only add memories slowly fills up with contradictions.
The replace primitive lets the system invalidate an old memory with a new fact or preference. This is how human memory works: you don't keep two home addresses for a friend, you update the one you had when they move.
What makes an AI memory system good?
The quality of a memory system comes down to two things:
- Classification. How good is the system at automatically identifying new memories that should be remembered, and spotting when one should replace an existing memory?
- Retrieval. How good is the system at recalling the right memories for what the user is trying to do right now?
If classification is weak, the system either misses the memories worth remembering or remembers irrelevant information.
If retrieval is weak, the right memory never shows up when it's needed, or the wrong one shows up instead.
Classification is a topic in its own right. If you missed it, I covered it in Build a better AI product with explicit classification. The same lesson applies here: an explicit decision about whether something is worth remembering is likely more reliable than prompting a model to handle memories itself.
There are many approaches to retrieval. A common one is semantic search, which finds memories with a similar meaning to the user's request rather than an exact keyword match. If you're an engineer, @tobilu/qmd is an npm package that makes this easy to get started with in TypeScript. Another technique is to look memories up directly in a database, using hard-coded rules about which memories each task needs.
Compression: text vs. structured memories
Because memories live in the context window, every memory costs tokens. So memories should be compressed when they're saved.
Memories can be compressed and saved in two ways: as generated text summaries, or as structured data with fields defined by your application.
For example, imagine a user prompts an AI recipe app with: "I can't have peanuts, and I've gone off cilantro too." The app's memory classifier picks up two memories.
If the app's memory system uses generated text summaries, the model writes a sentence for each memory:
- "User is allergic to peanuts."
- "User dislikes cilantro."
If it uses structured memories instead, the model populates its database with records like this:
| Type | Ingredient |
|---|---|
| Allergy | Peanuts |
| Aversion | Cilantro |
Structured memories offer several advantages over text memories:
- Clear business rules. A defined schema makes memories easy to inspect and query. A peanut allergy can be a hard filter on every recipe, rather than a hint the model might overlook.
- Easier replacement. If the user later says "cilantro's fine now", the system knows exactly which memory to replace: the cilantro aversion.
- Easier classification. The schema defines exactly which kinds of memory exist, so the model picks from a fixed set instead of inventing new phrasing each time.
- Easier retrieval. Memories are indexed in a database, so the app can run precise queries.
However, text memories have their own advantages:
- Flexibility. No schema or business rules are needed, which suits fuzzy domains (e.g., writing style), where preferences don't fit neatly into fields.
- Speed. They're faster to implement, with no upfront design required.
- Semantic retrieval. They pair naturally with semantic search, which finds memories by meaning rather than exact keywords or rules.
The two approaches aren't mutually exclusive. Many products use both.
Summary
As AI makes features easier to copy, memory is becoming one of the few lasting moats in software. Any memory system can be built on five primitives (remember, recall, search, replace and forget), but what sets a good one apart is how well it classifies what's worth remembering and retrieves the right memories at the right time.
Homework
Here are a couple of questions to help you bring memory and personalization into your own product:
- Where are your users repeating themselves? Those are your first opportunities for personalization.
- Text or structured memories? Consider which type of memory suits your application better.