It Counts. It Does Not Read.
Dashboards tell you where it hurts. They do not tell you what is broken. Four different organizational failures produce the same red tile, and the four fixes have nothing in common. I have sat in the meeting where the red tile got fixed and the failure did not.
I spent part of the summer on a method for telling them apart from data a company already has. It has only been tested in simulation, and it says so on every page. But the inversion at its center changed how I think about diagnosing any organization, and that part I am confident enough to write down.
The Inversion
The first design was an adaptive interview. A model asks the right people the right questions and narrows in on the mechanism. That is how a good consultant works, and it is what I assumed a good tool would do.
Working through it produced the opposite conclusion. The interview is the most expensive, most biased tool in the toolbox. People report perceptions. People have incentives. People do not know what they do not know. So the interview should run last and smallest, not first. Data should find the area and narrow it down, and the human conversation should be saved for the one question the data cannot answer.
If it runs past thirty minutes, the telemetry did not do its job.
Symptom Location Is Not Cause Location
The simulation ran a deterministic model of an organization with one planted fault and measured whether the method could name where the fault was and what kind it was, from team-level data alone.
The sharpest result: a broker fault, where one team sits between others and slows everything that crosses it, was localized correctly 6 percent of the time by a naive reading of the metrics and 100 percent of the time by a reading that used the dependency graph. The symptoms show up on the teams downstream. The cause is upstream. Without the graph, you fix the teams that are hurting, and nothing changes.
The result that surprised me: adding the graph made single-team faults harder to call, from 71 percent to 53 percent, by adding competing explanations at the same false-alarm rate. More sensors raise the evidence bar. More data is not free even when it is free.
Privacy Is Not a Setting
All data is team-level or coarser. Teams under five people roll up into their parent group at collection time, not at report time. That distinction is the whole design. Rolling up only at report time does not restore trust once people know the per-person data exists.
Count the teams of one before rolling them up, because an org chart full of them is itself a finding: key-person dependency, made official as boxes on a chart.
And never present a conclusion the data cannot support. "Indistinguishable from X without Y" is a valid and preferred answer. It tells the human exactly what the one remaining question is.
Why It Counts
The method counts. It does not read. That is a limitation and it is also the point. A count can be checked and proven wrong. A reading is a story, and stories about organizations are the easiest thing in the world to generate and the hardest thing to check.
Every mechanism I have watched fail inside a large company left a signature in telemetry the company already had. A queue that kept refilling. A team whose cycle time was fine and whose downstream teams' cycle times were not. A change that was reviewed by the same two people every time. The signatures were there. Nobody was looking at them as signatures, because the dashboard had already told them where it hurt and they went to fix the hurt.
I do not know yet whether the method survives contact with a real organization. No real-organization results exist and nothing here should be read as a claim about any actual company. What I know is that the bet underneath it is not that this kind of thinking is rare because it is hard. It is rare because nobody has packaged it, and the people who do it by instinct are the ones I have written about elsewhere, whose work looks like noticing things.
-- Justin Higgins. Software Engineer, Midwest. Built a diagnostic that counts, and put the interview last.
Companion pieces: The Work Looks Like Nothing - the people who notice. Survived Is Not Endorsed - the critic that checked the process.
Reactions, disagreements, war stories: jchigg2000.dev@gmail.com