How do you stop an AI's guess from quietly becoming "fact" in the next tool?

Discussions
,
1

@deanrie nudged me to spin this out of the “handling context across different AI coding tools” thread, so here it is.

Quick recap if you missed the other thread. Everyone’s been talking about pruning. Keeping memory small, deciding what’s worth storing. But there’s a different problem sitting underneath that nobody really named. Not “is this worth keeping” but “is this even true.”

Here’s the thing that kept biting me. After a few hops between Claude Code, Codex and Cursor, my memory had stuff the agent just made up and never checked, sitting right next to stuff I’d actually confirmed. Same shape, same weight, looks equally legit. Then one of those guesses gets pulled into a fresh session like it’s gospel, and the next agent quietly builds on it. That’s how cross-tool memory rots.

So the thing I’ve been building (piia-engram, local-first, open source) basically runs on three rules.

First, nothing the agent writes is trusted by default. It can’t promote its own stuff. It only becomes “confirmed” if I sign off, or if a hard signal does, like a passing test. No grading its own homework.

Second, “confirmed” isn’t one flavor. A test result is ground truth. Me saying yes is strong. The agent’s own reasoning is just a guess until something real touches it. Different sources, different weight.

Third, and this is the one I actually cared about. The tag lives in one local store that every tool reads through the same MCP server. Not plain text I’m praying each tool re-reads, not a separate memory per tool. One store, so the tag just travels with it. The catch, and I’ll be honest, it only works for tools that actually read that store. Anything walled off stays walled off.

Stuff I’d genuinely like to argue about:

  • where’s the line between “a human confirmed it” and “a test confirmed it,” and should they go stale at different speeds
  • how do you keep a “confirmed” fact from rotting after the world moves on (the classic “we use Jest” surviving a switch to Vitest)
  • for tools that just won’t read a shared store, is there an honest fix or do you just accept the walls

Repo’s in my profile if you want to see how it’s wired. Mostly I just want people to poke holes in it.

3

Glad you moved this into a separate thread. This topic deserves its own space outside the pruning discussion.

The most valuable thing in your approach, in my view, is the second principle: confirmed is not a single flavor. Most memory approaches collapse everything into a binary yes or no, and that is exactly where the rot you describe comes from. Provenance plus different weights by source is what is usually missing.

On your three questions, here’s what I think:

  • The line between a human-confirmed fact vs a test-confirmed fact, and how fast it expires. I’d separate them by nature, not just by weight. A test is a re-runnable truth. You can recheck it cheaply, so it doesn’t need a TTL, it needs a trigger. The fact is valid as long as the test is green. A human yes can’t be rechecked without asking a human again, so that’s what should decay over time. So the test doesn’t really expire slower, it’s tied to a signal, not to the clock.

  • We use Jest survives a move to Vitest. This is exactly the kind of case where TTL won’t help. The fact didn’t get old with time, it got invalidated by an event. It makes sense to attach facts to observable anchors, like a dependency or config existing in the repo, so when the anchor changes it automatically drops the fact from confirmed back to guess instead of waiting for a timeout.

  • Tools that don’t read the shared store. The honest answer is walls are still walls, there’s no magic here. But you can make the boundary explicit. Mark facts coming from non-reading tools as unverified on input. Then walled-off tools don’t silently poison the store, they go into the same guess until touched bucket as the agent’s own assumptions.

On the Cursor side, Rules and Memories cover some of this pain inside one tool, but they don’t have a trust and provenance layer like yours. So your approach is interesting. Drop the repo link in the thread and I’ll take a look at how the promotion logic works.

4

Thanks for the close read. You took this further than I had.

Your split is better than mine. Right now my decay is all time-based. Fresh, aging, stale, just by age. But you’re right. A test is something you can re-run. So it doesn’t need a timer, it needs a trigger. If the test is green, the fact stays true. Only a human “yes” should fade with time. I’ll split those two.

Same with Jest to Vitest. A timer is the wrong tool there. The fact didn’t get old. An event killed it. Better idea: tie a confirmed fact to something real in the repo, a dependency or a config line. When that thing changes, the fact drops back to “guess.” I’m adding this.

“Unverified on input” is a good call too. Right now I just keep outside stuff out of the store. Letting it in as “guess until checked” is more honest, and more useful.

Code is here: GitHub - Patdolitse/piia-engram: Local-first AI memory you can see, edit, and override — portable across Claude Code, Codex, Cursor, Windsurf, and other MCP coding tools. · GitHub

The part you asked about: an agent can’t set its own trust level. strip_untrusted_trust_fields in storage.py removes any tier it tries to give itself. To become “verified,” it has to go through staging review. Provenance and the time-based freshness are in provenance.py.

Two honest notes, since you’ll read the code. One, the trust gating is opt-in right now, not on by default. I need to tighten that. Two, the trigger and anchor ideas aren’t built yet. Your reply just named the gap.

This thread really helped. If you can point me to how Cursor Memories decides what to surface, I’d read it.

5

Thanks, I read it carefully. The foundation is solid: pure, stdlib-only, dicts aren’t mutated, so calling it on the read path is actually safe. No questions there.

But one structural issue outweighs everything else, and it comes straight from our earlier chat. We agreed that a test-confirmed fact is held by a trigger, and time-based fading is only for human-yes. Right now compute_freshness applies the same FRESH_MAX_DAYS and AGING_MAX_DAYS to any entry, without looking at the source. That means a green test fact will become stale after 90 days even though nothing broke. You already have resolve_source_agent and trust tiers, but freshness doesn’t use them at all. Until you close that gap, you can’t let freshness drive promotion or demotion because it’ll sink valid facts just because a timer ran out.

How I’d untangle it: freshness should check the source. Time decay should be only for human-confirmed. Test and anchor facts should be taken out of the time timeline entirely, either via a separate status like trigger_bound, or by simply skipping time decay. And I’d move the thresholds out of module-level constants into arguments for compute_freshness since once decay is source-aware, you’ll want different thresholds for different sources.

Smaller stuff for later:

  • _clean_identifier: the docstring promises not free text or paths, but in practice it only strips newlines and length. A path or short content will get through. If the goal is anti-injection, you need an explicit ban on separators.
  • Future timestamps are clamped with max(0.0, ...). That prevents crashes, but a garbage or skewed date will silently become fresh. I’d flag those instead of treating them as fresh.
  • annotate_freshness does a shallow dict(item). Nested provenance is still by reference. It’s fine now, but it’ll bite if someone starts touching nested data.
  • basis returns the string none, while status uses the constant UNKNOWN. Harmless, but make basis a constant too for consistency.

The fundamentals are right. The main thing is to tie decay to the source before freshness starts moving trust. Once you’ve got source-aware decay and anchor triggers, share it. I’d be interested to see it on real history.

6

This is really generous. Thanks for actually reading the code.

You nailed the main one. It’s the real hole. Right now compute_freshness is source-blind. It puts the same FRESH_MAX_DAYS and AGING_MAX_DAYS on everything. So a green test fact goes stale at 90 days even though nothing broke. resolve_source_agent and the tiers are right there. Freshness just never looks at them. So yeah, I can’t let freshness drive promotion or demotion yet. It would sink good facts on a timer.

Here’s what I’m taking from this.

  • time decay only for human-confirmed facts
  • test and anchor facts come off the time line completely, a trigger_bound status, not a clock
  • thresholds move out of module constants into args on compute_freshness, since once it knows the source you want different numbers per source

And the order matters like you said. Source-aware decay first. Then let freshness touch trust. Not the other way round.

The smaller ones are all fair.

  • _clean_identifier overpromises. It only strips newlines and length. A short path slips through. If it’s meant to block injection it needs a hard ban on separators. I’ll make the code and the docstring agree.
  • future timestamps quietly turning fresh is wrong. I’ll flag a skewed date instead of clamping it.
  • annotate_freshness only shallow copies. Nested provenance is still a reference. I’ll deep copy before anything touches nested data.
  • basis returns the string none. status uses UNKNOWN. I’ll make basis a constant too.

I’ll build source-aware decay and the anchor triggers, then bring it back here on real history like you offered. This gave me a much sharper target than I had. Thanks again.

7

I don’t really get the concept here. Am I understanding the problem you are trying to solve correctly? You are trying to deal with agent hallucination/mistakes/lies via a memory server system? That’s an interesting idea. I do think a good memory system will inevitably cut down on such problems since they are often caused by the agent nor remembering previous choices/actions. But I am unclear how you are trying to go about solving this problem. It seems a very open ended problem to try and run tests on saved data to confirm accuracy. I suppose if the project is very narrow this could be possible, like if a set of playwright tests, for instance, was all you were concerned with verifying. But, what if the type of information saved is broader, involved multiple projects, multiple testing systems, different architectures, etc? It seems like if you tried to somehow verify all that data, you would do not much else. If you don’t mind laying our your approach a little more, I am curious.

9

@neverinfamous, I think it’s worth rephrasing a bit, since there’s a small misunderstanding here. @Patdolitse, correct me if I’m wrong.

The idea isn’t to run tests against all saved data and “verify” it that way. That really wouldn’t scale. The point is provenance: every fact in memory carries a tag showing where it came from, and the trust level travels with that fact across tools.

Three different sources, three different modes:

  • agent guess: not trusted by default, and the agent can’t promote it by itself
  • human-confirmed: trusted a lot, but it fades over time (TTL), since you can only re-check by asking the human again
  • test or anchor-confirmed: tied to a trigger, not a timer. The fact is valid while the test is green or while the anchor (a dependency, a config line) is still there. If the anchor changes, the fact automatically drops back to guess

So nobody is “verifying everything” at runtime. For broad cases with lots of projects and architectures, this works via source tagging, not exhaustive testing. Expensive checks are only used where there’s a cheap, re-runnable signal.

So your point that “good memory reduces hallucinations” is right. This project just adds an extra layer on top: “how much can we trust this memory item at all.”

10

Please don’t take me as hostile. But I want to ask some hard questions so I can understand better. Maybe they aren’t hard questions and I just don’t grok. But, provenance sounds useful, for sure. If you can efficiently record who makes each and every change to code and even documentation, that sounds great, if you are a coder that modifies code yourself rather than always through agents. Then you could distinguish between code you added/changed and code the agent added/changed. You could do this with commit history, right? But, what if you jump in to modify/fix code the agent writes? Won’t that get confused? I guess I just don’t see how you are going to be able to label every piece of data saved in memory without that itself becoming a large amount of overhead and without introducing more chances of hallucination/misinformation. If the agent is relied on to do the labeling, what’s to prevent them from making mistakes/hallucinations in the process and thereby increasing error? Also, is the assumption that the human makes less mistakes than the agent? That may be but it seems unlikely if the human has to manually tag all these things. I would think the human would give up and just rubber stamp everything.

11

The three-tier model (agent guess → human-confirmed → test/anchor-confirmed) reframed how I was thinking about the problem, and it’s been rattling around in my head since.

Where it led me is this: what if you didn’t have to label trust at all?

My concern with any system where the agent assigns its own trust level is that the labels themselves become another surface for hallucination. You’re relying on potentially unreliable information to certify other potentially unreliable information. And if you put that burden on humans instead, the realistic outcome is rubber-stamping — especially across multiple projects.

What I keep coming back to is that most repositories already contain an immutable provenance record that nobody has to create or maintain. Git history tells you who changed something, when they changed it, and exactly what changed. That information exists whether humans or agents are doing the work, and neither party can fabricate it after the fact. A commit SHA is cryptographic. You can’t hallucinate one into existence.

So instead of trying to make each memory item carry its own trust score, I’ve been thinking about giving agents tools to answer targeted questions against that existing record:

  • Has this file changed since this memory was created?
  • Which commits relate to this issue?
  • Is this documentation describing the current state of the code?
  • Was this change made by a human or an agent?

None of that requires exhaustive verification. It’s cheap and targeted. An agent checking whether a piece of code has been modified since a journal entry was written is one query, not a full audit.

I recently began embedding commit SHAs directly into changelogs alongside agent-optimized descriptions. That way an agent reading a changelog entry can trace it back to the exact diff without any additional lookup overhead, and when it does need to dig deeper, the SHA is right there.

The broader picture I’m planning now thanks to this discussion is combining structured memory with git history mining — not to verify everything, but to give agents a way to detect when something they “know” might be stale, superseded, or contradicted by later changes. A lot of what we call hallucination is really just context drift and lost project history. Better memory helps with that, but access to objective provenance data would help more.

Your framing of “the fact is valid while the anchor is still there” is the piece that clicked for me. I’m just betting that for software projects, git is the most universal and lowest-overhead anchor available.

What do you think?

12

@deanrie — closing the loop. You said once I had source-aware decay and anchor triggers, share it on real history. It’s built. Here’s how it actually behaves.

Source-aware decay. compute_freshness now reads the source first, then picks a decay policy. Human-confirmed stays on the clock — it can only be re-checked by asking the human again. Test-confirmed, and anchors that still check out, come off the clock — trigger-bound, not timer-bound. One design call worth flagging since you read the code: I kept the public freshness_status as the same 4-state, time-based field for backward-compat, and made the source-aware part an additive signal (skip_decay / decay_policy) that the decay, refresh and stale-surfacing paths now honor. So a green-test fact still reports its real age, but the 90-day timer no longer pulls it into the stale/refresh/decay queues.

On real history: my store is ~400 entries. I’ll be straight with you — under the current labeling it’s 385 agent / 15 unknown, with no human/test/anchor-confirmed entries yet, because stamping is brand new. So this is the mechanism projected forward, not “my memory improved overnight.” Projected to +200 days, every entry goes stale as agent-sourced (correctly flagged for review). The same entries, if they’d been test- or anchor-confirmed, skip decay and stay out of the stale/refresh queues. Same entry, same age, only the source differs — which was exactly your point about freshness ignoring the source.

Anchors (the Jest→Vitest case). A fact can be tied to something observable in the repo — a dependency or a file — and a read-time check re-validates it against the actual manifests. On my real repo: anchored to dep:portalocker (present) → checks valid → skip_decay; anchored to dep:jest (absent) → checks invalid → drops straight back to time decay, i.e. back to guess. No timeout involved; the event invalidated it. The project binding is the normalized git remote (github.com/owner/repo), not the local root path, so it travels across machines (file anchors stay repo-relative). If the entry’s project doesn’t match the repo you’re checking it’s skipped untouched rather than guessed at; an anchor it genuinely can’t verify falls back to time decay — it won’t flip something to invalid on a miss.

The four smaller things you flagged are all in: the identifier cleaner now rejects path-shaped identifiers and unsafe separators (still allows things like github:actions); skewed/future timestamps are flagged instead of silently turning fresh; deep-copy before anything touches nested provenance; and basis is a constant now.

Two honest limits. One, I’m deliberately keeping freshness advisory — it drives decay/refresh proposals (owner-confirmed, dry-run by default), but I’m not letting it auto-promote/demote trust yet, even though the gap’s closed. I’d rather keep a human/governance step there for now. Two, anchor validation is owner-run and local/best-effort today — you run a check, it reads the repo — not a background watcher. And agents still can’t set their own trust through the agent-facing writes; trust/provenance fields are stripped there, and only the owner or an explicit internal path can stamp. That last part is also my answer to the “who does the labeling, and what stops the labeler from hallucinating” worry.

@neverinfamous — your git-history angle is the right instinct, and it overlaps with where this landed. I’m already using the git remote as the portable identity and a dependency/config as the observable anchor. Your version — query the record directly (has this file changed since the memory was written, commit SHA in the changelog) — is lower-overhead than my dep/file check and fits deanrie’s “cheap, re-runnable signal” exactly, just pointed at provenance instead of correctness. The one thing I’d watch: git tells you something changed and by whom, not whether the remembered conclusion is still true — a file can change without invalidating a lesson, and stay identical while the lesson rots from a dependency two repos over. So I’d treat git as a strong staleness trigger, not a truth oracle. As the universal, zero-maintenance anchor for “this might be stale, go re-check,” though, I think you’re right.

Thanks both — this gave me a much sharper target than I had.

13

Right architecture. freshness_status should stay 4-state, and skip_decay / decay_policy should be additive. Age and decay should stay decoupled. A green-test fact shows its real age, but it shouldn’t end up in stale queues.

On anchors, falling back to time decay on an unverifiable miss is correct. But anchor invalidated (dep:jest removed) and anchor unresolvable (couldn’t check) are different signals. The first is a clear staleness event and should drop to guess immediately, not go into generic decay. Don’t let the second hide the first.

385 agent / 0 confirmed is your real cold start. Early on, rely on anchors as the main promotion path, and keep human-yes rare. Bootstrapping via humans is just rubber-stamping.

Git as a staleness trigger, not a truth oracle, is still the best framing in the thread. Bring numbers when you have real anchor entries running on live dependency changes.

14

@deanrie — adopted the split, and I ran it on real deps this time instead of on paper. Widened the checks too, since this is easy to get subtly wrong.

When an anchor comes back invalidated (the dep or config it was tied to is actually gone), the fact now drops straight to a guess in the owner-run recheck. Tier goes to staging, the anchor confirmation is cleared, and the anchor metadata (ref, project id, status=invalid, checked-at) stays on as evidence of why it dropped. Unresolvable is a different case (unsupported manifest, an indirect -r/-c requirement, a ref it can’t parse). That just falls back to time decay like before, so “I couldn’t check” never gets read as “it’s gone.” compute_freshness is still a pure read. Only the recheck writes.

For the real run I used this repo’s own base deps. Two true facts, one anchored to portalocker (the JSON store write lock), one to mcp (the transport) as a control. Both confirmed against the repo’s real project id, both off the clock. Then I dropped only portalocker and re-ran:

both deps present -> both off the clock
drop portalocker, recheck:
portalocker  -> invalid -> demoted to a guess (staging, source cleared, status=invalid kept)
mcp (control)-> valid -> untouched, still off the clock
report: valid:1 invalid:1 demoted:1

One dep leaving doesn’t drag the rest down with it.

Recovery is one-way, on purpose. I put portalocker back and re-ran, and the demoted fact does not come back on its own. It isn’t an anchor entry anymore, so the recheck skips it. A dep showing up again doesn’t re-verify what the fact claims, so trust shouldn’t come back for free. You re-confirm it by hand, and even that only re-binds the anchor, it doesn’t push the tier back to verified. Auto-healing on a flapping dep is the false confidence I’m trying to avoid in the first place.

I checked it from a few sides, not just one. Lessons, decisions and playbooks all demote the same way, dep: and file: anchors both, a mixed batch only drops the invalid one, and there’s the recovery above.

Straight about what this is: a controlled run. Real project id, real dep file, the real resolver and recheck path. But the fact is seeded and I pulled the dep by hand, so it isn’t weeks of organic data. My store is still basically a cold start with nothing anchor-confirmed, so the organic numbers come later as real anchors build up against real dep churn. And on the cold start you’re right: anchors do the promoting, human-yes stays rare. Git as a staleness trigger, not a truth oracle. That’s the frame.

15

This is the version I wanted to see. The invalidated vs unresolvable split is now where it should be: a missing dep drops straight to guess, while “I couldn’t check” falls back to time decay and never gets read as “it’s gone.” That distinction was the main thing from my last note, and keeping compute_freshness a pure read with only the recheck writing keeps the boundary clean.

Two design calls I’d specifically endorse:

  • Keeping the anchor metadata (status=invalid, checked-at) on the demoted fact as evidence. You want to know why something dropped, not just that it did. That’s what makes this auditable instead of magic.
  • One-way recovery. This is the counterintuitive but correct one. A dep reappearing doesn’t re-verify what the fact claims, so trust shouldn’t come back for free. Auto-healing on a flapping dep is exactly the false confidence the whole system exists to prevent. Re-binding the anchor on manual re-confirm without pushing the tier back to verified is the right amount of friction.

Including mcp as a control in the run was a good instinct too. The property that actually matters is isolation. One dep leaving doesn’t drag the rest. A control is the cheapest way to show it.

On what’s left: you’ve named it yourself. Seeded fact, hand-pulled dep, so this proves the mechanism, not behavior under real churn. The interesting failures show up there. Watch the cases where one dep leaves and another arrives, like the literal Jest to Vitest move. Does that read as a plain invalidation, or do you eventually want to detect the successor rather than just dropping to guess? Same question for transitive or indirect deps and monorepo workspaces, where “present in the manifest” gets fuzzy.

Bring it back when you’ve got organic anchor entries surviving real dependency changes over a few weeks. Git as a staleness trigger, not a truth oracle. That framing has held up across the whole thread.

16

@deanrie — thanks, that genuinely means a lot. The successor case (jest→vitest) is the one I keep turning over too: today it reads as a plain invalidation, but detecting the successor instead of just dropping to guess is the more interesting version, and transitive/monorepo deps are exactly where “present in the manifest” stops being a clean yes/no. No point theorizing without data though, so I’ll let real anchors ride actual dependency changes for a few weeks and come back with what actually happened.

17

@Patdolitse @deanrie — appreciate the continued rigor here, and I want to be upfront about where our approaches actually diverge, because I think it’s a genuine design difference rather than one being strictly better.

What you’re solving: A general-purpose trust layer for agent memory. The core insight is that not all stored facts are equally reliable, and the source of a fact should influence how long it’s trusted and what it takes to re-certify it. The invalidated vs. unresolvable split, one-way recovery, and the anchor-as-observable-truth concept are all sound engineering answers to that problem.

Where I see the friction: The system’s value is gated on human participation that tends to degrade over time. Your own numbers said it — 385 agent / 0 confirmed at cold start, and bootstrapping via human labeling is rubber-stamping. The tier model works well once it has data, but it depends on a pipeline that, in practice, many projects won’t sustain. The labeling surface is also additive complexity that agents themselves could corrupt, even with the trust-field stripping mitigation.

What we’re doing instead: memory-journal-mcp takes a narrower bet — that for software projects, the dominant memory failure mode isn’t wrong information (trust failure) but stale information (staleness failure). We’re planning a git history tool group (get_commit, search_commits, get_release_commits, get_unreleased_commits, analyze_commit_patterns) backed by the existing simple-git dependency. The idea is to give agents targeted query capability against objective provenance without requiring any labeling step — has this file changed since this entry was written, which commits relate to this issue, does the changelog match the current state of the code. A SHA in a changelog entry is zero-maintenance provenance: it’s cryptographically immutable and requires no human to certify it.

We’re also exploring a resource (not yet implemented) that would compare git commit history directly against live files to surface staleness signals automatically — closer to your anchor concept, but driven by the objective record rather than declared anchors.

Honest limits of our approach:

  1. Git tells you something changed, not whether the remembered conclusion is still valid. A file can change substantially without invalidating a lesson. A dependency two repos over can rot a lesson without touching any tracked file. You named this already — git as a staleness trigger, not a truth oracle. That’s the right framing and we’re keeping it.

  2. The successor problem (jest→vitest) is a real gap. Git surfaces that jest left the manifest. It doesn’t tell you the lesson attached to jest should now be read against vitest. That translation still requires either a human or a smarter semantic layer or journal entries for context.

  3. Our approach is strongest for software projects with active git histories. For domains without that record — research notes, long-horizon planning, personal knowledge — the provenance anchor disappears.

The two ideas from this thread I’m actually planning to carry back: the invalidated/unresolvable distinction as a signal quality model (not just “stale” but why stale), and the one-way recovery principle — an entry demoted for staleness shouldn’t automatically regain significance when conditions temporarily look favorable again. Good thread. The successor detection and transitive dep cases are the ones I’d most want to see real data on.

20

Fair pushback @neverinfamous — let me take the “if I didn’t know I was lied to,
nothing could be done, any tagging would be based on the same lies” part head-on,
because it sounds fatal but it’s actually the exact spot the default-unverified
setup is for.

Back to the default-unverified setup I described — I don’t have to know it was a
lie. Nothing an agent writes is trusted to begin with, so there’s no moment where
I’m judging “true or false” and could get fooled into mis-tagging it. A confident
wrong inference and a correct one land identically — both unverified — until
something external promotes one: a sign-off, a passing test, a live repo anchor.
Your end-of-thread summary is a real safety net, but it leans on you catching it
at close-out; this just never hands the unnoticed lie “fact” status in the first
place, so there’s nothing left to catch.

The “untrue stuff degrades and gets pruned anyway” line is the one I’d actually
push on, because the failure that bit me does the opposite. The wrong belief that
hurts isn’t the obscure one — it’s the one referenced constantly, which is exactly
what importance-scoring keeps. Anchored facts dodge the whole question: kill the
dep and the fact demotes itself back to a guess, no significance heuristic in the
loop.

Anyway — I finally got this from “thing I keep describing in here” to “thing you
can actually run.” pip install piia-engram, open source, all local JSON you own
(nothing leaves the machine). No pressure at all, but if you ever feel like kicking
the tires: onboard a repo, accept a couple facts, recall them in another tool, and
it shows you why each one’s trusted. You’ve run cross-tool memory longer than
almost anyone here, so your “here’s where it breaks” would mean a lot.

21

@Patdolitse, nice wrap-up. Your post brings out what’s really the core idea of the whole system, and it’s worth saying it out loud.

The objection “if I was tricked without noticing, then any tag is built on the same lie” sounds fatal, but default-unverified simply sidesteps it. You don’t need to catch the lie. A confidently wrong conclusion and a correct one land the same way, both are unverified, until something external promotes them. So there’s no point where you make a “true or false” call and can mislabel it. That’s stronger than any model that requires detection, and it also answers @neverinfamous’s worry that “the labels themselves are a new surface for hallucinations.” If the agent can’t promote itself, a label mistake doesn’t increase trust, it stays in the same guess-until-touched bucket.

And your pushback on the “untrue stuff degrades and gets pruned anyway” argument lands. A dangerous wrong belief isn’t the obscure one — it’s the one referenced all the time, which is exactly what importance scoring keeps. That’s the failure mode that bites in practice. Anchored facts take the whole question off the table: if a dependency is gone, the fact demotes itself back to a guess, without any significance heuristic in the loop.

On the split with @neverinfamous, I wouldn’t read it as “who’s better.” You’re addressing two different failure modes: trust failure, the fact is wrong, and staleness failure, the fact is outdated. Git as an objective provenance record is an almost zero-maintenance staleness trigger, and declared anchors give source-aware trust where git is silent. They add up, they don’t compete. And both approaches hit the same frontier, the successor case like jest to vitest, transitive or monorepo deps, where “present in the manifest” stops being a clean yes or no. That’s where real data gets interesting.

So the plan stays the same. Let the organic anchor records live through real dependency churn for a couple weeks, then bring numbers. “Git as a staleness trigger, not a truth oracle” held up through the whole thread, and I think it will keep holding. You had a good conversation.

22

Hey, I am working though a ton of changes to memory journal right now, some of which are based on this conversation. I was hoping to wrap everything up before responding but didn’t want to leave you hanging, so to speak. I added a confidence score throughout the server and now have the agent setting a confidence and significance score on entries and session summaries based on some criteria instead of a default number for significance. This seems to be working very well in regard to significance scores and the numbers look good so far with the confidence scores. I am also filtering the session briefings themselves now by the significance score so if entries or session summaries don’t meet the threshold they aren’t included.

I also have an autonomous pruner set up through the scheduler that operates regardless of the timeframe set for auto-pruning, though I need to test it some more. It seems fine. And, I have the MCP prompt to optimize the database. The agent has some scripts to help with this if the database is large but I have found mine is small enough to simply have them manually optimize the whole thing-pruning, tags, relationships, etc. It takes less than 5mns and produces fantastic results. Because I am constantly testing the server, test artifacts collect that I have to clean up constantly. Yes, I could switch to a test database but I am working on several projects simultaneously and don’t want the data for them going to a test database. It works but maybe I will just add a test database to the system so I don’t have to worry about it. I

I am not yet pruning based on confidence because I want to see how those scores do first for a while. Fundamentally, I have low confidence in agents confidence levels. They just lack common sense and judgment, as we all know. But, so far the numbers look reasonable. I still have to do the detailed planning for the new resources to check for staleness and a few other related things but I will get back to you. I also consolidated the prompts and resources, cut some that I originally built for it that aren’t useful enough to warrant the token expense, and converted some of my key workflows to prompts, before adding the new resources. Now, I am ready to look at the new resource(s) which are mostly focused on the trust issue you are trying to solve but honestly I am unsure about it.

This is a bit off-topic but I also built a Headless Execution Runner/Agent-to-OS Bridge. If I were to give it a formal technical description, it is a Hardened Subprocess Orchestrator built specifically for AI agents. It makes agent terminal operations far more reliable and faster than the terminal in Cursor/Antigravity/etc.

It’s Not Just a CLI: While it is invoked via the command line (bun agent-exec.ts), its purpose isn’t to be used by humans. It doesn’t have a traditional human-friendly interface (like interactive prompts or standard flags). It strictly consumes JSON payloads.

  1. It’s a Bridge: It acts as a literal translation layer between an Agent’s intent (the JSON payload) and the Host Operating System (node:child_process). It bridges the gap between our environment (which requires completely non-interactive, headless execution) and standard developer tools (which constantly try to spawn interactive editors, TTY shells, and blocking prompts).
  2. It’s an “Orchestrator/Runner”: Because it handles the lifecycle of the execution. It manages the environment variables, stream buffers, backpressure, timeout timeouts, and process tree termination.

Essentially, it is the protective middleware that allows an AI agent to safely wield native developer tools (like docker, git, and pwsh) without getting trapped in interactive shells or crashing the system with unbounded output.

We also switched to a git-history single source of truth philosophy. We aren’t maintaining a changelog/unreleased file during work, just the git history. I will create a changelog for the users at release time probably, but they have the release notes we make for them, the readme, and the wiki. The git history is for the agents. It works well but I built this CLI designed specifically for autonomous agents (and CI/CD systems) to deterministically query, filter, and extract git history as structured JSON. I’m really excited to replace the changelog information in the session briefing with git-history but I am still dealing with edge cases and stuff in the tool and don’t want to have to update it later with the final logic.

Here is why it is tailor-made for AI agents:

  1. Machine-Readable by Default: It outputs structured JSON or JSONL formats instead of raw text, automatically mapping complex semantic data like Conventional Commits, Fixes #123 issue links, breaking change descriptions, and custom git trailers into a heavily typed schema.
  2. Deep Guardrails: Features like 100MB buffer chunking, UTF-8 safe patch truncation, limits on how many files it will parse per commit, and strictly catching dangling EPIPE streams are all designed to prevent a hallucinated or broad query by an AI agent from crashing the runtime or hanging the terminal.
  3. Semantic Filtering: Instead of relying on an agent to properly escape complex shell pipes (e.g., git log | grep ...), it provides high-level CLI flags (--changelog-only, --breaking, --impact, --category) that let agents search git history semantically as if it were a database.
  4. Single Source of Truth (SSoT) Alignment: It acts as the backbone for workflows where git history is the actual documentation (like a living changelog). It gives agents a reliable, read-only interface to accurately reconstruct the timeline of a project.

It’s essentially a specialized bridge that translates the messy, raw output of git log into a clean API that an agent can ingest without fear of crashing its context window.

I may integrate both into memory-journal-mcp but these have much broader uses. I will look at your code when the planned changes to memory-journal are complete and before final testing.