Back to blog
July 21, 2026·5 min read·Metronix Team

One Embedding Space Can't Hold Everything Your Agent Needs to Remember

A single vector index with one decay curve cannot tell a durable preference from a one-off event. Typed records fix that at write time.

One Embedding Space Can't Hold Everything Your Agent Needs to Remember

Most agent memory setups still push every fact into the same vector index with the same decay curve: a support ticket from last year, a Slack message from ten minutes ago, a user's stated preference, all embedded the same way and aged the same way. That's the default because it's the easy thing to build first. It's also the wrong default once an agent runs long enough to accumulate real history.

The flat vector store is a shortcut, not an architecture

One embedding model, one index, one similarity search — you can stand this up in an afternoon, and for a demo it works fine. The problem shows up once the store has more than a few weeks of history in it. Similarity search doesn't know that "the user prefers async standups" and "the deploy failed at 3am Tuesday" are different kinds of facts. One is a durable preference that should stick around until explicitly changed. The other is an event that's mostly useless a day later. Retrieval treats them identically because, to the index, they're just vectors with a timestamp.

What splitting by type actually buys you

The fix isn't a smarter embedding model. It's writing a type on every memory record and giving each type its own write path, retention rule, and retrieval logic instead of one global decay function.

A rough split that holds up in practice:

  • Episodic — events, conversation turns, one-off observations. Short shelf life. TTL-based expiry is fine here; nobody needs the exact wording of a status update from three weeks ago.
  • Semantic — facts, preferences, entity attributes. These don't decay gradually, they get replaced. "Account owner is X" isn't 60% true after six months, it's either still true or it's been superseded by a new fact.
  • Procedural — how-to knowledge, runbooks, established patterns. Mostly static, updates are rare and deliberate.

This isn't a novel take MM invented — it matches where the field has been moving. A 2026 architecture survey on agent memory describes production stacks as "not a single architecture but a small stack: vector memory for fast fuzzy recall, an episodic buffer for short-term coherence, and a graph for entity-heavy queries" with the agent routing between them (digitalapplied.com, 2026). Zep's approach models facts with valid_at/invalid_at timestamps instead of a decay score, which is the semantic-memory case above stated more precisely (referenced via mem0.ai's 2026 state-of-agent-memory writeup).

Decay is the wrong tool for facts that get superseded

Decay assumes staleness is smooth — relevance fading a little more each day. That's a reasonable model for episodic memory. It's the wrong model for facts. A fact doesn't get 5% less true per week. It's correct until something contradicts it, then it's wrong, full stop.

Treating supersession as decay is how you end up with an agent that's 70% confident about a customer's plan tier when the real answer is "the tier changed eight days ago and the old record just hasn't faded out yet." What you actually want at write time is a check: does a new candidate fact conflict with an existing record on the same subject and predicate? If yes, the old one gets marked superseded, not slowly discounted.

The harder problem is the write path, not the read path

Most writing on agent memory focuses on retrieval — hybrid search, rerankers, chunking strategy. Fair enough, retrieval is where the demo lives. But the failure mode that actually breaks production systems happens earlier, at ingest.

If the write path doesn't check for supersession before accepting a new record, you get slow accumulation of near-duplicate facts about the same entity, each with a slightly different embedding, all still in the store. Retrieval then does exactly what it's supposed to do: it returns several of them, and the agent gets handed contradictory context. Nobody notices in week one because there's nothing to contradict yet. It shows up three to six months in, when an agent confidently states two different things about the same customer in the same conversation and someone has to go figure out why.

Worth designing the classification-and-supersession check into the write path from day one rather than bolting it on after the first contradictory-answer bug report.

Where to look if you want to dig in

If you're browsing metronix-memory and want to see this in code rather than take it on faith: look for the memory-type classification logic that runs at write time, and the freshness/supersession checks that decide whether an incoming record replaces an existing one or gets appended alongside it. Those two pieces are where this design decision actually lives, as opposed to the retrieval layer where most memory-system write-ups spend their time. If you find a case where a fact type is missing or the supersession check misses an obvious duplicate, that's a legitimate first PR.

Are you splitting write paths by memory type already, or running one embedding pipeline and hoping decay quietly covers for it? And if the latter — what was the first contradictory-answer bug that made you reconsider?

References

  • Atlan, "Agentic AI Memory vs Vector Database: Architecture Guide 2026" — https://atlan.com/know/agentic-ai-memory-vs-vector-database/
  • mem0.ai, "AI Agent Memory 2026: Progress Benchmark Report Evaluations" — https://mem0.ai/blog/state-of-ai-agent-memory-2026
  • Digital Applied, "Agent Memory Architectures: Vector vs Graph vs Episodic" — https://www.digitalapplied.com/blog/agent-memory-architectures-vector-graph-episodic
  • "Rethinking Memory Mechanisms of Foundation Agents in the Second Half: A Survey" — https://arxiv.org/pdf/2602.06052