The notes that were never quite true

A few months ago, a note in my vault almost talked me into a real decision. It was articulate. It was specific. It was beautifully formatted, with a confident little table and a number sitting right in the middle of it. And the number was wrong.

It wasn’t wrong because anyone lied. It was wrong because it was a summary of a summary of a summary, and somewhere four generations up that chain a figure had drifted, the way a story drifts when it’s retold. By the time it reached me it had lost all memory of where it came from. It sat in the same folder, in the same font, with the same calm authority as the bank statement two notes over. Nothing on its face said I am a guess about a guess. That was the whole problem.

I keep what people now call a second brain: a few thousand Markdown files — meeting notes, transcripts, scanned documents, half-finished ideas, and a steadily growing pile of things an AI summarized for me. An assistant sits on top and retrieves from it when I ask a question. And for a long time that assistant did something quietly dangerous: it treated every file as equally true.

But they are not equally true. Some of my notes are ground truth: a contract, a law, an ID card, a raw record, a figure I checked at its source. Some are first-order syntheses an AI wrote by reading three other notes. And some are syntheses of syntheses, three or four hops removed from anything real. An AI synthesis can be fluent and completely wrong, and when the retriever pulls that fluent-wrong note into an answer, the error launders itself into “fact.” Confidence is not evidence. My files had no way to tell the two apart, so neither did I.

The fix was not a smarter model. It was provenance — making every note declare, on its own face, where it came from and how much it has earned the right to be believed.

A pedigree on every note

Every substantive note now carries six lines of metadata at the top:

• tier — how many hops it is from raw reality. Tier 0 is primary: an original document, a contract, a law, an ID, a raw email straight from the sender. Tier 1 is a mechanical one-to-one transform of a single primary — a cleaned-up transcript, an OCR’d scan. Tier 2 is a first synthesis. Tier 3 pulls several sources together. Tier 4 and up is synthesis-of-synthesis. The rule is brutally simple: a note’s tier is one more than the highest tier it was built from.
• reliability — a number between 0 and 1, full precision, no rounding to neat tenths. Primaries are set by hand. A contract, a law, an official document, an ID, a raw record, is 1.0, because a document stays what it is whether or not I remember it right. My own first-hand account sits a notch below that, because memory is not evidence and I can be wrong about the very things I am surest of. A raw transcript is 0.99, almost perfect, but a microphone can mishear. A scrape off the open web is 0.8.
• sources — links to the exact notes this one was derived from. These links are the edges of a trust graph. No sources means it’s primary.
• volatility — static, slow, or fast. Is this a fact that will be true forever, or a price that’s stale by Friday?
• verified — the last date a human actually confirmed the claim. This one line, it turns out, carries most of the signal.
• generated_by — who or what produced the text: me, a transcription model, a specific LLM, or a mix. Orthogonal to everything else, because how a sentence was written is a different question from how true it is.

The arithmetic of doubt

Reliability does not get added or subtracted as knowledge moves through the system. It gets multiplied, and the size of the multiplier depends on what kind of step was taken.

A mechanical copy loses nothing and keeps all of its source’s trust. A faithful summary loses almost nothing and keeps about 98 percent. A genuine first synthesis, where a model reads a few notes and draws a conclusion, keeps about 96. A weave of several sources keeps 95, and a speculative leap that runs ahead of what the sources actually say keeps about 90. The steep drops are reserved for the steps that genuinely add risk, not for the honest ones. An earlier version punished every hop by the same flat amount, and trust collapsed far too quickly. Tying the penalty to the kind of inference is what fixed it.

What still holds is weakest-link, never average. A synthesis is only as trustworthy as the shakiest thing it leaned on, because an average would let one solid source paper over a rotten one. And because each hop multiplies rather than subtracts, trust slides toward zero without ever reaching it, so even a deep chain leaves every link with a distinct, comparable score instead of bottoming out into a meaningless flat zero.

Decay is only half the story, though, and the better half is that trust is also allowed to rise. A claim that three independent sources agree on is not a guess, it is close to settled, and the system now treats it that way: corroboration pushes a note’s credibility up toward certainty rather than dragging it down. A well-supported conclusion can hold its ground across many hops instead of fading just because it travelled far. That is why the model keeps two separate numbers, not one. Source reliability is the track record of where something came from. Claim credibility is how well the specific fact is confirmed. They are different questions and they deserve different answers. And underneath all of it sits a floor: the least trusted note is at most halved, never zeroed. The system demotes. It does not delete.

Volatile facts get a second discount: a freshness clock. Effective reliability is the stored reliability times one-half raised to (age ÷ half-life). A price has a 90-day half-life — believe it now, distrust it by autumn. A salary band or a regulation gets a year. A birthday never decays at all. And underneath all of it sits that one human field, verified: the date I last looked the fact in the eye and nodded. No amount of clever math beats a recent human confirmation, and the system is honest enough to admit it.

The trust scores that did nothing for months

Here is the part I’m least proud of. For a long stretch, none of this metadata did a single thing.

Every note had its tier and its reliability and its honest little verified date sitting right there at the top, and the search index my assistant actually queries was throwing all of it away. It read the words and ignored the pedigree completely. I had built an elaborate system of trust and then quietly filed it under decoration. The retriever went on ranking notes purely by how well the text matched, which is exactly the behaviour that lets a confident fourth-hand summary win in the first place. The labels were a conscience the machine couldn’t hear.

Fixing that was the work of the last few days, and it’s the part that finally made the whole thing real. The vault is turned into a searchable index by a process called vault_indexer.py — part keyword, part meaning. The change was to stop ranking on relevance alone and start ranking on relevance times trust times freshness times tier, with the trust multipliers applied after the semantic search has done its part. Each class of note now carries a weight: a raw session log sits at 0.55, an extracted lesson at 0.80, a project or person record at 1.00, a distilled skill or principle at 1.20. The whole behaviour lives behind one switch, VAULT_TIER_WEIGHTS_ENABLED, so I can turn it off and measure whether it earned its place instead of trusting my own taste about it.

The effect is small to look at and large to live with. A trustworthy note now outranks a more eloquent one even when the eloquent one is the better word-match. The retriever has, in a real and switch-flippable sense, learned to down-weight its own past guesses. The most articulate thing in the pile stopped winning by default. The most trustworthy one started to.

The graph that maintains itself

There’s one more piece, the one that keeps the whole thing from rotting, and it’s the part I’m wiring in now. A note’s truth is only as current as its sources. So a nightly pass walks the sources edges of every derived note and asks a simple question: has any source changed since the day this note was last verified?

It answers that with content hashes, not modification times — because my files sync across machines constantly, and sync rewrites timestamps for a living. A hash doesn’t lie. When a source’s hash changes after a derived note’s verified date, that derived note gets flagged stale, automatically, and the reverse dependency edges are recomputed by the script, never by me. The dependency graph keeps its own books. I just get told when something I once trusted is now standing on shifted ground.

What I deliberately did not build

The most interesting decisions here were the refusals.

I do not store a separate trust score per dimension. There are no confidence intervals. There’s no Dempster-Shafer fusion, no EigenTrust, no learned weight on every individual edge of the graph. I pressure-tested the design against a panel of background agents whose entire job was to argue for more machinery — more signals, more fusion, more cleverness — and the verdict came back the same every time: the pedigree and the date carry the signal, and everything past that is bureaucracy wearing the costume of rigor. One uniform retention factor. Weakest-link. A freshness clock. A human verified stamp. That’s the whole engine, and it earns every part it keeps.

What it actually is

Stripped of the mechanics, what I built is a knowledge base that knows what it doesn’t know. Every note can tell you not only what it claims, but how it came to claim it, how many hands it passed through on the way, how far it drifted from anything real, and whether it’s quietly gone out of date since the last time anyone checked.

The assistant sitting on top is calmer for it. It has stopped mistaking its own eloquence for proof. When it answers me now, the ground truth speaks first and the confident guesses know their place.

It turns out the most useful thing I ever gave my second brain wasn’t more memory.

It was doubt.