The Shape Leaks
In April I wrote a working paper with a thesis I still think is beautiful, and in August I ran the experiment that killed it. This is the whole arc, because the killing is the part worth reading.
The Idea
More and more, architectural code review runs through a language model. A senior engineer once spent days tracing layering violations across a service boundary. That engineer now pastes a module into a chat window and gets a diagnosis in seconds. For an organization whose code is the product, and for anyone in healthcare or finance or defense, that is not an option. The code cannot leave the tenant, and in-tenant models lag the frontier by a generation. The choice on offer is weak diagnosis on private code or strong diagnosis on exposed code.
I proposed a third option. Do not redact the code. Do not encrypt it. Translate it into a different domain. A membership-management service becomes a logistics dispatcher. A derivatives pricing library becomes a chess engine. The god-objects, the circular dependencies, the layering violations, the copy-pasted error handling, the hidden state machines all survive the translation, because the sickness is in the structure, not the words. Ask a frontier model to review the chess engine, and its diagnosis holds for the source, because the sickness survived.
I called the translated artifact a husk. The invariant that made it work was that the husk had to keep the structure a diagnosis depends on: the same code property graph within a tolerance, the same anti-pattern density, the same mix of comment types. And the invariant had to be enforced by a verifier, not asked for in a prompt. A rewriter's training rewards refactoring, and it will quietly improve your code if you let it. Prompt-only enforcement of that invariant is a category error.
That last part held up. The rest did not.
The Test I Wrote Before I Knew the Answer
The paper said the privacy claim, that a consumer model cannot work out the source domain from the husk, had never been tested. It was the next thing to test. So I pre-registered the test. Six synthetic domains, three held out. An attacker panel of two models. A twelve-option menu asking which domain a husk came from, and a free-text version. A blind arm shown only a structural stub, as the floor. The source code itself, unhusked, as the ceiling. Four husking methods in between. The pass criterion, written in advance: the attacker's accuracy on husks must not be distinguishable from its accuracy on the blind stub.
It ran on August 19. An account spending cap halted it mid-flight. It resumed under the same run identifier and finished: 986 of 998 attacker calls, twelve lost in transport and counted as missing rather than scored wrong.
All four methods failed.
Three of them were at the ceiling. Format-preserving encryption of identifiers, literal tagging, and a composition of both: the attacker identified the domain from the husk 100 percent of the time on the menu, the same as from the raw source. Those methods did nothing, which the paper had predicted.
The LLM translation, the one the paper was actually about, was the only method doing any work. It roughly halved every leakage measure. On the menu the attacker got 78 percent against 33 percent blind. In free text, about 50 percent against zero. The 95 percent confidence interval on the husk-minus-blind difference was positive at both ends. It failed the bar I had set, and it was the only thing in the experiment that moved the needle at all.
The Floor Was Not a Floor
One of the amendments I had to write afterward is the kind of thing that makes me distrust every accuracy number I read.
The blind attacker, shown a ten-number structural stub with no code in it, answered the same domain on 26 of 29 items and 23 of 30 items regardless of what it was shown. Its 33 percent was not weak structural signal. It was response bias. Hold each attacker's answers fixed, shuffle the true labels ten thousand times, and the blind arm's corrected score is about plus 0.03, which cannot be told apart from nothing. The husk-minus-blind difference barely moves, so the failures stand. But "33 percent floor" should never again be described as a floor. The stub carried nothing.
The Finding That Ends the Premise
Then I ran the experiment I did not want to run.
Take each source file and canonicalize it mechanically. Remove every identifier, every string literal, every comment, every piece of visible text. Leave the control flow, the call graph, the arity, the nesting, the coupling exactly intact. Eleven hard gates enforce that no text survives, and three model-free probes confirm no token is concentrated in any domain. Then ask the same attacker panel the same twelve-option question.
Corrected accuracy: about plus 0.23 to plus 0.26. The blind arm is plus 0.03. The full husk is plus 0.44 to plus 0.48. The source is plus 0.67.
Shape alone carries roughly half the leak a husk carries. Canonicalizing every numeric value too left the score unchanged or slightly higher, so it is not constants leaking. It is control flow, call graph, arity, nesting, and coupling.
That is the premise eating itself. The technique requires keeping the architecture, because the architecture is where the sickness lives. And the architecture itself tells you the domain. The thing I am required to keep is part of what leaks, and no verifier enforcing the invariant I wrote can fix that. The honest form of the claim is a trade-off, keeping the structure against keeping the domain private, rather than a promise of both.
That is going into the paper as a status note at the top rather than a rewrite of the body. The body is preserved as written. The limitations file is the one that governs, and it says: read this first.
What It Does Not Say
Some caveats are load-bearing and I will not bury them.
Recognition is not recall. The attacker chose from a menu of twelve with the right answer on it. In free text, the translation removed two thirds of the signal. Quoting the menu number alone overstates the leak.
The structure-only run carried no source control arm and no blind arm of its own, so its formal validity gate is recorded as failed. Its numbers come from the permutation baseline, which needs no blind arm, and should be read as that and nothing more. The open-ended endpoint was not run against it.
And the corpus is synthetic. No real organization, product, or regulator exists in it. The threat model names identifier-level re-identification, the company, the product line, the proprietary algorithm. Nothing here can test that, for any arm, because of how this was built. The only specific identifier ever recovered from any husk was the corpus's own invented module name, which turned out to be a shipped defect in the rename step: it survived in 22 of 90 husks, and on those the attacker was right 44 of 44 times. Fixed now. The remaining eight leaks arrive as content, a heading in a rendered page, with no import to get the name from. No rule based on imports can see those.
One more channel, and it is embarrassing. A tree whose every file body had been faithfully rewritten into an unrelated domain still announced its source through the filesystem. Twenty-three occurrences of one source-domain term across the paths. You could get the domain from a directory listing alone, without opening a single file. The experiment harness deliberately closes that channel, which is flattering to the husk. I say so in the limitations, because a reader who did not know would overrate the result.
Copying does not explain the leak either. The share of copied spans was 0.269 when the attacker was right and 0.276 when it was wrong. One method leaked at 100 percent while copying nothing. The leak survives a genuine rewrite.
What I Keep
The category error still stands: you cannot enforce an invariant on a rewriter by asking it nicely. The verifier is still the load-bearing piece, it just cannot buy what I wanted it to buy. And LLM translation is still the only thing that reduces domain signal, by about half, which is not nothing for a threat model where the attacker is a curious model rather than a determined adversary.
Mostly what I keep is the experiment. I wrote the pass criterion before I had a result. I wrote the amendments when the floor turned out to be bias. I ran the version that could only hurt me. The paper's own last section is titled "What would falsify this," and it did.
I was not happy about it. I like the idea. I still like the idea, which is the problem with ideas. I still think there is something in it, in the trade-off form. But the version in the April paper is dead, and I killed it, and I would rather say that here than have someone else find it in the limitations file.
-- Justin Higgins. Software Engineer, Midwest. Pre-registered the test that would kill his own paper, then ran it.
Companion pieces: Twelve Days of Green Tests - the anchor you did not author. Survived Is Not Endorsed - the losers are evidence too.
Reactions, disagreements, war stories: jchigg2000.dev@gmail.com