Your Agent Can Remember. Can It Notice?
Everyone building agent memory is optimizing retrieval. The valuable signal is somewhere else entirely.
Last weekend I was standing over a golf ball well below my feet, and a playing partner offered some advice: “I think the ball is going to go left off that lie.”
The reaction arrived before the reasoning did. Hmm. No. Below my feet, that ball’s going right.
I didn’t look anything up. I didn’t consciously retrieve the physics. Ball below your feet, the face tilts right of the target, the ball goes right, every time, for as long as I’ve played this game. What happened was faster than recall: his claim hit something I knew, failed to reconcile, and my brain escalated the conflict before I could name the rule behind it.
That’s a different capability than memory.
Retrieval answers a question you already know to ask. Noticing tells you there’s a question you should be asking.
Then I looked at my own agents and realized none of them could do this.
the setup
I run a lot of agents across a number of domains: pricing, news, filings, relationships to name a few. Each has spent months building its own institutional memory in local markdown files, with interlinked notes, source references, accumulated context, domain-specific assumptions.
The substrate is where it should be. The files are portable, inspectable, and vendor-free. Each vault is good. The problem is that they’re good alone.
The pricing agent knows a name by its ticker. The news agent knows it by the company name. The relationships agent knows it by the people who work there. Three agents, three labels, one underlying entity, and no layer that knows they’re discussing the same thing.
So when the relationships vault learns something the news vault’s assumptions depend on, nothing happens.
Nothing happening is the failure.
It’s not a hallucination, a crash, or a bad retrieval result. It’s a failure of attention. One part of the system already knows that another is operating on a stale assumption, and nothing surfaces the conflict. Downstream analysis keeps running as though the new information never arrived.
what everyone optimizes instead
Most discussion of agent memory is about retrieval quality, embedding models, context management, recall benchmarks, and increasingly elaborate ways to place the right fact into the model’s context window.
That work matters. But it treats memory as a storage-and-retrieval problem: put facts in, get facts out, measure the fidelity.
The files aren’t forgetting the facts. The system is failing to connect them.
Consider what happens when two knowledge sources disagree. A last-write-wins architecture replaces the old belief with the new one. The stale assumption disappears, and so does the fact that a meaningful conflict ever existed. An append-only architecture preserves both versions in history, but the disagreement stays merely discoverable, and only to someone who already knows to look.
Neither reproduces the brain’s most useful trick: escalating a conflict before you’ve consciously retrieved either fact.
Neither notices.
contradiction as an event
The layer I’m building above these vaults treats contradiction as an event, not a state.
Each agent keeps writing only to its own markdown files. A rebuildable graph sits above them, resolving shared entities and exposing cross-domain relationships. The files remain the source of truth; the graph is just the index that makes the connections visible. If it corrupts, it rebuilds from frontmatter, wikilinks, aliases, and source metadata. Nothing important lives only in the graph.
When a new edge conflicts with an existing one, the system doesn’t overwrite the old belief and doesn’t silently file the new one next to it. It writes to a queue.
The contradiction queue is the most valuable view in the whole setup, not because it holds more information than the vaults, but because it identifies where the information no longer fits together.
Agreement between domains is useful, but it’s usually confirmation. Contradiction means one of three things: one agent is stale, one agent is wrong, or the world changed and only one of them has noticed.
All three are exactly what I want surfaced.
Agreement is confirmation. Contradiction is information.
Here's what an entry looks like. The values are mocked; the structure is real.
CONTRADICTION 2026-07-21
entity: Meridian Standard Corp (MRSD)
relationships-vault:
"CFO (our primary contact) departed the firm on 2026-06-30"
source: relationships/mrsd-contacts.md
news-vault:
"Q2 coverage summary lists the same CFO as the current management team"
source: news/mrsd-q2-coverage.md
corroboration:
relationships claim supported by 2 current sources
news assumption supported by 1 older source
action:
escalated for reviewTwo lines like these can sit in separate folders for weeks. Both retrievable. Neither hidden. Until the system resolves them to the same entity, nothing knows they disagree.
And the consequence isn’t just one stale markdown file. Analysis, summaries, and anything downstream can keep operating on an assumption another part of the system already knows is false.
the hmm has a threshold
There’s a second behavior worth borrowing from human attention: the strength of the response scales with the strength of the belief.
You don’t get the internal alarm every time new information conflicts with something you vaguely remember. You just update and move on. The reaction is strongest when the contradiction hits something you know. My playing partner’s read didn’t survive contact with a few hundred sidehill lies.
Agent memory should work the same way.
Edges in the graph carry weight: how recently a claim was updated, how many independent sources support it, how many domains corroborate it, how consequential it is downstream. Contradicting a weakly supported fact is a silent update. Contradicting a heavily corroborated belief stops the line.
Look back at the queue entry. The departure claim carries two current sources; the coverage claim carries one stale one. A relationship vault that logs every meeting accumulates weight, and by the time the contradiction fires, one side of it is heavily corroborated and the other is a summary somebody wrote a quarter ago. That’s an escalation, not a coin flip.
That’s the humans-on-top pattern applied to memory. Most updates resolve autonomously. A human doesn’t approve every write or adjudicate every minor discrepancy. But disagreements that cross a confidence threshold get surfaced explicitly.
No approval theater on every change. No silent resolution of the conflicts that matter.
If you’ve been reading a while, this is the widest gap again, one layer down. There, disagreement between opinionated agents marks where judgment matters. Here, disagreement between domains marks where attention goes.
Same signal, different layer.
the boring part, as usual
The graph isn’t the hard part. Graphs are easy.
The hard part is entity resolution.
The system needs a registry that knows a ticker, a company name, the people attached to it, and a shorthand label can all refer to the same underlying object. It’s what let the queue entry fire at all: the vaults had to agree that our contact and the CFO in the coverage note were the same person before anything could notice they disagreed. Without that registry there are no reliable cross-domain edges. Without cross-domain edges there are no contradictions to catch. The system goes blind to everything outside each individual vault.
Identity is the boring infrastructure of memory.
Nobody’s excited to build the alias registry. Unfortunately, everything depends on it.
Most agent-memory systems are trying to become better filing cabinets: store more, retrieve faster, stuff more relevant material into context. Useful, but incomplete.
A filing cabinet gives you the document you ask for. A colleague tells you the document you’re reading doesn’t match what the team learned three months ago.
Memory that retrieves is a filing cabinet. Memory that notices is a colleague.
If you run more than one agent with more than one knowledge base, go find two files that disagree about the same entity. They exist. The question is whether anything in your system knows it.