Storing an AI’s memories is the easy half. The hard half is getting the right one back at the right moment — and that’s where two very different approaches show up, each with a failure mode the other doesn’t have.
Vector search: retrieval by meaning
Vector search converts text into embeddings — long numeric vectors that encode meaning — and finds stored memories whose vectors sit closest to your query’s. The useful consequence is that matching happens on meaning rather than wording. Ask about “the login problem” and you can surface a memory that only ever said “authentication failure,” because those two phrases land near each other in embedding space.
That property is genuinely powerful, and it’s why nearly every AI memory system uses vector search as its backbone. It handles the common case well: you half-remember something, describe it loosely, and get back the relevant context.
Where it breaks down: vector search retrieves whole chunks based on their overall semantic character. So a specific fact buried inside a long memory about something else is easy to miss. A memory that’s mostly about a deployment pipeline, which happens to mention a colleague’s name in one sentence, has an embedding dominated by deployment-pipeline meaning. Query for that person and the chunk may not surface at all — the fact is stored, it’s just not findable by similarity.
This is a structural limitation, not a tuning problem. Averaging a passage into one vector necessarily washes out details that don’t match the passage’s dominant topic. You can mitigate it with smaller chunks, but smaller chunks fragment context, which costs you on the conceptual queries vector search is actually good at. It’s a real tradeoff, not a free win.
The other well-known issue: vector search always returns something. Ask about a topic with nothing stored, and you get the nearest neighbors anyway — which may be only loosely related. Relevance is relative, not absolute, so the system can’t easily tell you “nothing here matches.”
Knowledge graphs: retrieval by relationship
A knowledge graph takes the opposite approach. Instead of embedding text, it extracts entities — people, projects, services, URLs, tickets — and the relationships between them, storing those as an explicit structure. This person works on that project. This service depends on that database. This bug was fixed by that commit.
Retrieval then becomes traversal: start at an entity, follow relationships, collect what’s connected. The result is exact rather than approximate. If a fact was extracted into the graph, you get it back precisely, every time, regardless of how the surrounding prose was worded.
That directly addresses vector search’s blind spot. The colleague’s name buried in the deployment memory becomes its own node, connected to the project — retrievable by relationship even though it was semantically invisible.
Where it breaks down: graphs only know what was successfully extracted into them. Extraction is imperfect, and anything the extractor misses is simply absent — a silent gap, since a traversal that finds nothing looks the same as a fact that never existed. Graphs also handle fuzzy queries poorly: “what were we thinking about the caching strategy” has no clean entity to start from. And they carry real maintenance cost — schemas drift, extraction quality varies, and a graph that goes stale is worse than no graph, because it returns confident wrong answers.
Why the answer is usually both
Laid side by side, the tradeoff is clear:
| Vector search | Knowledge graph | |
|---|---|---|
| Best at | Fuzzy, conceptual queries | Precise, factual lookup |
| Matching | Approximate, by meaning | Exact, by relationship |
| Fails when | A specific fact is buried in a larger chunk | The fact was never extracted |
| Handles novel phrasing | Well | Poorly |
| Maintenance cost | Low | Meaningful |
They fail in almost exactly opposite directions, which is what makes combining them attractive. A hybrid approach runs both retrieval paths and merges the results: vector search covers “what does this generally relate to,” graph traversal covers “give me the exact fact I need, reliably.” You get conceptual recall without giving up precise lookup.
The catch is that hybrid isn’t free. You’re maintaining two retrieval systems, and you need a merge strategy — how to rank and deduplicate results that arrive from paths with incompatible notions of relevance. Done carelessly, a hybrid system is just two mediocre retrievers producing noisier output than either alone.
How to think about choosing
If you’re building or evaluating an AI memory system, the practical questions are:
What kind of recall failures actually hurt you? If your pain is “it didn’t connect two related conversations,” that’s a vector search problem. If it’s “it doesn’t remember my company’s name even though I’ve said it ten times,” that’s the buried-fact problem a graph solves.
How much structure does your domain really have? Codebases, teams, and infrastructure have dense, explicit entity relationships — good graph material. Freeform notes and open-ended discussion have less, and the graph earns its maintenance cost less clearly.
Can you measure it? The honest answer to “does adding a graph improve recall” is domain-dependent, and it’s worth testing on your own data rather than assuming. Define what a successful retrieval looks like, run the same queries through both paths, and compare — before committing to the cost of maintaining two systems.
Vector search is the right default; it’s cheap and it handles the common case. A knowledge graph is what you add when precise, reliable fact retrieval matters enough to justify the extra machinery — and for memory systems that need to reliably know who and what and where, not just about what, it usually does.
How that hybrid path is used in the MCP loop is in How MCP-Based Persistent Memory Works. The failure modes that remain even with both paths are in How AI Memory Retrieval Fails: Similarity, Recency, and Contradictions.
Start your 14-day Nexus-Catalyst trial to use hybrid retrieval on your own project context.