Calafia blog

🧠 AI agent memory: keyword, semantic, or extracted?

Ask an agent whose notes say "lot full by 7am" whether parking is tight, and you find out which memory provider it runs. Under keyword ranking, that note never surfaces — "parking," "tight" and "lot full" share no words. Under semantic ranking, it surfaces easily, because the sentences mean the same thing. Same stored notes, different selection, different answer.

That example is the whole axis in miniature. Agent memory has two halves — where notes are stored, and how they are ranked into the next run's context — and the ranking half is the one that changes behavior. Storage is a database decision. Ranking decides what the agent notices.

The rubric

Comparing memory providers honestly needs three questions:

  1. What retrieval failure does it fix? (Not "is it smarter" — which specific miss goes away.)
  2. What does retrieval cost per run, in money and in determinism?
  3. What happens when it breaks? Every provider fails eventually; the degrade path is part of the product.

The three providers, as they exist here

This deployment's registry carries three — the same list the memory axis page renders live.

Firestore keyword (the default). Ranks stored notes by word overlap with the current goal, plus small boosts for recency and for notes you saved explicitly. Free, deterministic, debuggable: you can read a memory and predict whether it will be recalled, which is worth more during development than any accuracy gain. Honest limits: the parking example above is its signature miss — vocabulary drift between how a note was written and how a goal is phrased. Best for: starting; staying, if your agents' notes and goals share a vocabulary.

Firestore semantic. Embeds the goal and the notes (Gemini text-embedding-004) and ranks by cosine similarity. Fixes exactly the vocabulary-drift miss — that is the one failure it reliably repairs, and the honest reason to switch is having actually seen that failure. Two design details worth knowing because they remove the usual migration pain: vectors are backfilled lazily (a note without an embedding still competes, ranked by overlap, and gets its vector cached in passing — so switching needs no migration and works on the first retrieve), and both Firestore providers read the same store, so switching never strands or loses data. Honest limits: it costs an embedding call per retrieve, it is not byte-reproducible, and if the key is missing or the embed call fails, it degrades to overlap ranking with a logged warning — an agent that recalls less, never a dead agent. Which also means a misconfigured semantic provider looks like a working keyword provider. Check the logs, not the vibes.

Mem0 (bring-your-own-key). The first external provider through the same contract: a hosted service that extracts facts from what it's given and stores the distillate, rather than ranking your raw notes. It appears as an option only for workspaces that have added their own Mem0 key, and memories are scoped so one workspace's can never reach another's. Honest limits: it's a network dependency with someone else's ranking logic inside; extraction means the stored fact is the service's paraphrase, not your note; and every Mem0 error degrades to empty results with one warning, because an external vendor outage must never break an agent. Best for: teams already invested in mem0 who want their existing memory layer under an agent, not a first choice.

Verdict: start on keyword, and let a miss you can name move you. The upgrade path is real but it is pulled by failures, not pushed by features.

A note on what "memory" excludes here, because adjacent axes get blended into the word: material handed to a run from outside (its own prior reports) is the retrieval axis, and the single trusted memo an agent leaves its next run is the state axis. Memory is specifically the accumulating, competing notes — the ones a ranking policy has to choose between. The competition is what makes the provider choice matter.

"Config compatibility is not recall compatibility"

The three providers accept the same configuration surface and the same store. That makes switching safe. It does not make them interchangeable: each one recalls a different subset of the same notes, which is a behavior change you will see in the deliverable, not in any error log. Treat a provider switch like a model switch — something you evaluate, not something you assume.

Which provider actually moves a recall-dependent task's eval score — and whether mem0-style extraction beats plain semantic ranking — is the measurable version of this post, and we have not measured it yet. The memory axis is one of the five the experiment runner sweeps: same task, one provider per cell, the agent's own eval held fixed. That sweep is queued as a data post; whatever it shows, including "no difference," publishes. Until then this post claims mechanisms, not magnitudes.

Where this leaves you

Situational advice: default keyword; switch to semantic the first time you catch the agent missing a note that was plainly relevant but differently worded; consider mem0 only if you already run mem0. Whatever you pick, the choice is per agent, not per account — a recall-heavy agent and a stateless reporter have no reason to share a provider.

What this doesn't tell you: how your notes are phrased relative to your goals, which is the entire variable. Ten minutes reading your agent's actual saved memories will predict the right provider better than any post. Written August 2026; the registry this describes is rendered live at /plugins/memory, and when the two disagree, trust the page.

The reframe: memory isn't a bigger context window on a delay — it's a selection policy, and the policy is the product. Set it per agent at /u/build, or sweep it on an agent you already have at its Tuning page.

Make it yours →