Verify the Artifact, Not the Process
I open-sourced a tool this summer that reads your entire Claude Code history off disk and turns it into a dashboard: spend, agent fleet, a scroll-driven retelling of every project, full-text search, and a page that mines your own corrections out of your own transcripts. It's what the Python script I wrote about in June grew into.
Before it went public I ran three audits on it. The third one found two guards that looked like they already worked, and both were checking the wrong side of a boundary. I want to walk through them, because the fix that mattered is a general rule I'd never said out loud to myself.
The Screenshot Script
The README has screenshots, and they have to come from a synthetic corpus, because the real one is my entire working history. The Repos page prints each repo's absolute path on its card, so a screenshot of the real thing would put my home directory and every project name into a committed PNG.
So the screenshot script had guards. It checked that a fixture corpus existed, checked that the server could serve it, and printed green.
It was shooting the real corpus. Every PNG in my README draft was a screenshot of my actual life, nicely cropped.
The page works out which corpus to show from localStorage, seeded by whichever npm script currently holds the port. Run the dev server, then run the screenshot script, and the browser loads the real seed while the script verifies the fixture seed. The script had verified what the server could serve and never checked what the browser would render.
The fix has three parts, and only one of them matters. The script now writes the corpus into localStorage itself. It refuses a corpus that resolves inside the repo. And it refuses to write any PNG whose rendered text contains the capturing machine's home directory.
That last check is the one that doesn't depend on my having guessed the failure right. The first two fix the bug I found. The third fixes the bug I haven't found yet, because it looks at the artifact and asks whether it contains the thing I never want in an artifact, however it got there.
The Fixtures Script
The second one was worse in a smaller way. The fixture generator recursively deleted whatever directory an environment variable named. The relocated-repos branch of the script had a marker-file guard, but the corpus root never got one. Set the variable to your Documents folder by accident and it eats Documents.
I verified the fix by pointing it at a real directory and watching it refuse. I didn't just read the code. I ran the dangerous thing against something I cared about and confirmed it wouldn't.
Your Localhost Tool Is a Network Service
The same audit produced the rule I now apply to every local tool I build.
The dashboard binds to 127.0.0.1 and has no auth, because it reads your filesystem and only you should be able to reach it. But loopback isn't the boundary. The browser is. Any page open in your browser can make a request to a localhost port, and whether it can read the response depends on defaults you didn't set.
Vite reflects any localhost or 127.0.0.1 origin back in its CORS headers by default. That means a page served by any other dev server on your machine could read your whole transcript corpus through mine. I pinned CORS off on both the dev and preview servers and verified it by request, not by assumption.
The API is connect middleware installed in Vite's server hooks, which Vite runs before its own DNS-rebinding host check, so that check never covered my routes. A page that rebinds its own hostname to 127.0.0.1 on my port would read everything as same-origin. The API now rejects any request whose Host header isn't loopback with a 403. Every route is a read, so anything other than GET or HEAD gets a 405. The two routes that take a file path check it against the root they were given, following symlinks while they do it, and a link inside the root that points outside is rejected rather than read.
I found the same thing on my own in a different repo two weeks later: a control plane with open-by-default writes on a loopback port, which a page in the operator's browser could drive as a simple cross-site request. Same conclusion from the opposite direction. Loopback binding isn't a browser boundary.
What the Refuted Finding Taught Me
The third audit raised 20 findings and 19 survived an adversarial check. The one that was refuted is the one I think about.
An old revision of a source file contained the string "job-search-engine." The finder flagged it as a leaked private project name. It was actually the previous scrub's replacement token, the sanitizer's output rather than its input. A second auditor, looking for leaks, had found the evidence that the first scrub worked and read it as evidence that it hadn't.
That's a second-order failure, and I don't have a rule for it yet, except this one: you have to know what your own tools leave behind, or you'll keep finding them.
The History Rewrite
After the audit I rewrote git history for the third time. The roadmap, the decision log, the instruction file, and the entire agent scratch directory came out of every commit. An old revision of the roadmap held my real spend figures for the year. A private repo name and an internal tooling reference came out of blobs and commit messages, including one commit subject that read "drop a private name," which is its own small joke. Sixty-two commits became sixty-one when one went empty. I deleted the GitHub repo and recreated it so no pre-rewrite object is fetchable by SHA, and verified that all four old SHAs return 422.
The local files still exist on disk, gitignored. That's the shape of every one of these fixes. Nothing was destroyed. The boundary between what exists and what ships just got a check on the shipping side.
And Then Nobody Came
I'll end with the part I'd normally leave out.
The repo went public: sixty-one commits, 149 tests, MIT license, secret scanning, push protection, branch protection. Uptake over the window was zero views, zero forks, zero stars, zero watchers, and two clones, one of which was almost certainly the CI runner.
The roadmap says so. Its previous entry had said the repo was still private and flipping it was left to me, and the GitHub API said otherwise. It had been created public and went private some time later, and GitHub fires no event for going private, so the duration isn't recoverable. I wrote a rule under that: never trust this file's claim about remote visibility over the API.
Your documentation isn't a sensor, and neither is the guard that prints green. Only the artifact is.
-- Justin Higgins. Software Engineer, Midwest. Shipped a tool that reads his whole history, and checked the screenshots for his home directory.
Companion pieces: The Work Looks Like Nothing - the script this tool grew out of. Everything Looks Like Everything - the part that fails is the part nobody thought to check.
Reactions, disagreements, war stories: jchigg2000.dev@gmail.com