It Counts. It Does Not Read.
Dashboards tell you where it hurts, not what's broken. Four different organizational failures produce the same red tile, and the four fixes have nothing in common. I've sat in the meeting where the red tile got fixed and the failure didn't.
I spent part of the summer on a method for telling them apart from data a company already has. It's only been tested in simulation, and it says so on every page. But the inversion at its center changed how I think about diagnosing any organization, and I'm confident enough in that part to write it down.
The Inversion
The first design was an adaptive interview, where a model asks the right people the right questions and narrows in on the mechanism. That's how a good consultant works, and it's what I assumed a good tool would do.
Working through it led to the opposite conclusion. The interview is the most expensive, most biased tool in the toolbox. People report perceptions, they have incentives, and they don't know what they don't know. So the interview should run last and smallest, not first. Data should find the area and narrow it down, and the human conversation should be saved for the one question the data can't answer.
If it runs past thirty minutes, the telemetry didn't do its job.
Symptom Location Is Not Cause Location
The simulation ran a deterministic model of an organization with one planted fault and measured whether the method could name where the fault was and what kind it was, from team-level data alone.
The sharpest result: a broker fault, where one team sits between others and slows everything that crosses it, was localized correctly 6 percent of the time by a naive reading of the metrics and 100 percent of the time by a reading that used the dependency graph. The symptoms show up on the teams downstream, and the cause is upstream. Without the graph, you fix the teams that are hurting, and nothing changes.
The result that surprised me: adding the graph made single-team faults harder to call, from 71 percent to 53 percent, by adding competing explanations at the same false-alarm rate. More sensors raise the evidence bar. More data isn't free even when it's free.
Privacy Is Not a Setting
All data is team-level or coarser. Teams under five people roll up into their parent group at collection time, not at report time, and that distinction is the whole design. Rolling up only at report time doesn't restore trust once people know the per-person data exists.
Count the teams of one before rolling them up, because an org chart full of them is itself a finding: key-person dependency, made official as boxes on a chart.
And never present a conclusion the data can't support. "Indistinguishable from X without Y" is a valid and preferred answer, because it tells the human exactly what the one remaining question is.
Why It Counts
The method counts. It does not read. That's a limitation, and it's also the point. A count can be checked and proven wrong. A reading is a story, and stories about organizations are the easiest thing in the world to generate and the hardest to check.
Every mechanism I've watched fail inside a large company left a signature in telemetry the company already had: a queue that kept refilling, a team whose cycle time was fine and whose downstream teams' cycle times weren't, a change reviewed by the same two people every time. The signatures were there. Nobody was looking at them as signatures, because the dashboard had already told them where it hurt and they went to fix the hurt.
I don't know yet whether the method survives contact with a real organization. No real-organization results exist, and nothing here should be read as a claim about any actual company. What I do know is the bet underneath it: this kind of thinking isn't rare because it's hard. It's rare because nobody has packaged it, and the people who do it by instinct are the ones I've written about elsewhere, whose work looks like noticing things.
-- Justin Higgins. Software Engineer, Midwest. Built a diagnostic that counts, and put the interview last.
Companion pieces: The Work Looks Like Nothing - the people who notice. Survived Is Not Endorsed - the critic that checked the process.
Reactions, disagreements, war stories: jchigg2000.dev@gmail.com