Verify the Artifact, Not the Process
I open-sourced a tool this summer that reads your entire Claude Code history off disk and turns it into a dashboard. Spend, agent fleet, a scroll-driven retelling of every project, full-text search, and a page that mines your own corrections out of your own transcripts. It is what the Python script I wrote about in June grew into.
Before it went public I ran three audits on it. The third one found two guards that looked like they already worked. Both were checking the wrong side of a boundary. I want to walk through them, because the fix that mattered is a general rule I had never said out loud to myself.
The Screenshot Script
The README has screenshots. The screenshots have to come from a synthetic corpus, because the real one is my entire working history. The Repos page prints each repo's absolute path on its card. A screenshot of the real thing would put my home directory and every project name into a committed PNG.
So the screenshot script had guards. It checked that a fixture corpus existed. It checked that the server could serve it. It printed green.
It was shooting the real corpus. Every PNG in my README draft was a screenshot of my actual life, nicely cropped.
The page works out which corpus to show from localStorage, seeded by whichever npm script currently holds the port. Run the dev server, then run the screenshot script, and the browser loads the real seed while the script verifies the fixture seed. The script had verified what the server could serve. It never verified what the browser would render. Every guard printed green and every PNG was of my actual life.
The fix has three parts and only one of them matters. The script now writes the corpus into localStorage itself. It refuses a corpus that resolves inside the repo. And it refuses to write any PNG whose rendered text contains the capturing machine's home directory.
That last check is the one that does not depend on my having guessed the failure right. The first two fix the bug I found. The third fixes the bug I have not found yet. It looks at the artifact and asks whether it contains the thing I never want in an artifact, no matter how it got there.
The Fixtures Script
The second one was worse in a smaller way. The fixture generator recursively deleted whatever directory an environment variable named. The relocated-repos branch of the script had a marker-file guard. The corpus root never got one. Set the variable to your Documents folder by accident and it eats Documents.
I verified the fix by pointing it at a real directory and watching it refuse. Not by reading the code. By running the dangerous thing against something I cared about and confirming it would not.
Your Localhost Tool Is a Network Service
The same audit produced the rule I now apply to every local tool I build.
The dashboard binds to 127.0.0.1 and has no auth, because it reads your filesystem and only you should be able to reach it. Binding to loopback is not a boundary. The browser is. Any page open in your browser can make a request to a localhost port. Whether it can read the response depends on defaults you did not set.
Vite reflects any localhost or 127.0.0.1 origin back in its CORS headers by default. That means a page served by any other dev server on your machine could read your whole transcript corpus through mine. I pinned CORS off on both the dev and preview servers. Verified by request, not by assumption.
The API is connect middleware installed in Vite's server hooks, which Vite runs before its own DNS-rebinding host check. So that check never covered my routes. A page that rebinds its own hostname to 127.0.0.1 on my port would read everything as same-origin. The API now rejects any request whose Host header is not loopback with a 403. Every route is a read, so anything other than GET or HEAD gets a 405. The two routes that take a file path check it against the root they were given, following symlinks while they do it. A link inside the root that points outside is rejected rather than read.
I found the same thing on my own in a different repo two weeks later. A control plane with open-by-default writes on a loopback port. A page in the operator's browser could drive them as a simple cross-site request. Same conclusion from the opposite direction: loopback binding is not a browser boundary.
What the Refuted Finding Taught Me
The third audit raised 20 findings and 19 survived an adversarial check. The one that was refuted is the one I think about.
An old revision of a source file contained the string "job-search-engine." The finder flagged it as a leaked private project name. It was not. It was the previous scrub's replacement token. The sanitizer's output, not its input. A second auditor, looking for leaks, had found the evidence that the first scrub worked and read it as evidence that it had not.
That is a second-order failure and I do not have a rule for it yet. Except this one: you have to know what your own tools leave behind, or you will keep finding them.
The History Rewrite
After the audit I rewrote git history for the third time. The roadmap, the decision log, the instruction file, and the entire agent scratch directory came out of every commit. An old revision of the roadmap held my real spend figures for the year. A private repo name and an internal tooling reference came out of blobs and commit messages, including one commit subject that read "drop a private name," which is its own small joke. Sixty-two commits became sixty-one when one went empty. I deleted the GitHub repo and recreated it so no pre-rewrite object is fetchable by SHA, and verified all four old SHAs return 422.
The local files still exist on disk, gitignored. That is the shape of every one of these fixes. Nothing was destroyed. The boundary between what exists and what ships just got a check on the shipping side.
And Then Nobody Came
I will end with the part I would normally leave out.
The repo went public. Sixty-one commits, 149 tests, MIT license, secret scanning, push protection, branch protection. Uptake over the window: zero views, zero forks, zero stars, zero watchers, two clones, one of which was almost certainly the CI runner.
The roadmap says so. The previous entry in the roadmap had said the repo was still private and flipping it was left to me, and the GitHub API said otherwise. It had been created public and went private some time later, and GitHub fires no event for going private, so the duration is not recoverable. I wrote a rule under that: never trust this file's claim about remote visibility over the API.
Your documentation is not a sensor. That is the general shape of everything in this post. The guard that prints green is not a sensor either. Only the artifact is.
-- Justin Higgins. Software Engineer, Midwest. Shipped a tool that reads his whole history, and checked the screenshots for his home directory.
Companion pieces: The Work Looks Like Nothing - the script this tool grew out of. Everything Looks Like Everything - the part that fails is the part nobody thought to check.
Reactions, disagreements, war stories: jchigg2000.dev@gmail.com