The Audit Trail

Guardrails, logging, scoring, and provenance in an agent-driven development workflow — and how to install the same discipline in any harness.

Tag legend

The provenance marks used throughout the system, as they appear in roadmaps, decision logs, and code.

Evidence over recollection

AI agents now write most of the code in this workflow. That speed is only safe if every claim about the work can be traced to a record.

Agent-driven development produces a large volume of machine-authored code, requirements, and documentation. Without discipline, three questions become unanswerable: who decided this, when, and on what evidence? The system described here makes those questions cheap to answer. It rests on one principle:

Four pillars implement the principle:

• Logging. Every working session is recorded verbatim, archived automatically, and searchable forever.
• Scoring. Quality-critical work ships only when it measurably beats the best prior version — wins and losses kept in append-only ledgers.
• Instrumentation. Cost, token, and activity telemetry at the session level; the same observability pattern built into every shipped app.
• Tagging. Every requirement carries provenance: human-decided user-ratified or machine-inferred CLAUDE-ORIGIN — ratified, or merely proposed.

Logging

The session record is the court of record. Everything else — git, docs, memory — is secondary evidence.

What it captures

• Every agent session is written to disk as structured JSONL: each prompt, each model reply, each tool invocation and its result, in order, verbatim. Currently 10,661 transcript files across 156 project directories (6.2 GB).
• A dedicated archiver (claude-vault, a small Rust CLI) imports transcripts into one SQLite database with full-text search. Today it holds 1,610 sessions and 295,422 messages in a 560 MB file.
• Archiving is wired into the session lifecycle: hooks fire the import when a session ends and before long conversations are compacted — capture survives even context truncation, with no human step.

How it is read

• A local web viewer (claude-code-log) provides full-text search across every transcript in every project, with highlighted snippets, newest first.
• The viewer is deliberately read-only: the audit surface cannot modify the record it audits. A load-bearing invariant, not a convenience.

The rule that makes it matter

Any investigation of why, who, or when starts at the session record — never at documentation. Git history answers only what changed and when.

Where it lives

Scoring

Quality-critical work ships through a ratchet: a scored competition against the best prior version. Ties lose.

How the ratchet works

• Two independent builds. Two agents implement competing versions in parallel from deliberately divergent starting skeletons.
• A divergence gate. Candidates that differ only cosmetically are rejected before scoring — the competition must explore real alternatives.
• A versioned rubric. Each candidate is scored per axis — correctness, architecture, tests, readability, robustness — under a rubric whose version is recorded with the scores.
• A strict gate. The winner ships only if its total strictly beats the reigning champion. Equal is not better; nothing ships on a tie.
• Losses are recorded too. Every run is appended to the ledger, win or lose. A failed attempt is audit data, not something to be tidied away.

The ledgers

38 quality ledgers exist today: per-feature ledgers committed inside each repo, plus a global cross-repo ledger for one competitive page family. Each row records run date, per-axis scores, total, rubric version, and result — so any shipped version can be compared against everything that lost to it.

Right-sized testing

Test suites are generated by one calibrated command rather than ad hoc, so suite size is a deliberate, budgeted choice — not agent enthusiasm. A repo with no tests is a decision on record — not a gap.

Where it lives

Instrumentation

The workflow measures itself at three layers: the session corpus, the harness lifecycle, and every shipped application.

Session-level telemetry

The viewer computes corpus analytics live from the raw logs: agent dispatch counts and fleet hours, token flow, prompt-cache utilization, and spend by day. Spend is estimated at public list prices per token and labeled as an estimate — it tracks billing closely but is never treated as an invoice.

Harness lifecycle hooks

The harness instruments itself through hooks that fire at session start, session end, before context compaction, after file edits, and on prompt submit. The archive import rides the session-end and pre-compaction hooks; policy reminders ride the others.

The application observability pattern

• Append-only event log plus a telemetry store (latency, tokens, cost) with audit events for logins and account changes.
• A no-op-safe emit seam: one emit module is the sole write path, wrapped so observability can never break the request that produced the event.
• Denormalized actor attribution: identity captured at write time — no join back to user tables that could later rewrite history.
• Admin-only aggregates with percentiles, and prune-on-read retention.

Ship-time guards

• The release pipeline refuses destructive git operations (--force, --no-verify, hard resets, forced branch deletion).
• Secret and oversized-file guards run before anything is staged.
• One attempt per mutating command — no blind retries. On any error the pipeline stops clean and reports exactly where every piece of work sits.

Tagging & provenance

Every requirement answers one question on sight: did a human decide this, or did a machine infer it?

Authorship tags

Any requirement an agent inferred — rather than the owner decided — is tagged CLAUDE-ORIGIN at write time. The tag is greppable; 323 files carry it today. At read time the rule inverts: agent-authored text is never cited back as binding without being re-derived first.

Approval classes

"Go ahead" is not a line-by-line ratification. Only the first class binds; the lower three never round up, and silence is never consent:

Decision logs & the single roadmap

• 45 repos carry an append-only DECISIONS.md: dated entries superseded by new entries, never rewritten.
• 48 repos carry a single ROADMAP.md — the sole source of truth for execution. Rival plan documents are folded in and deleted.
• Parked work is non-blocking by default; it can gate other work only through an explicit ⛔ BLOCKS line quoting the owner's own instruction.
• Sections are never renumbered, so citations stay stable; sessions hand off through a ▶ RESUME HERE block inside the roadmap itself.

Rules carry their own receipts

Standing rules are date-stamped with how each was earned — earned 2026-07-25 stated 2026-06-26 — often with the incident quoted verbatim. The governance layer obeys the discipline it enforces.

Instrumenting another harness

The pillars are portable. In dependency order — each step is useful on its own; together they reproduce the full audit trail.

Verification notes