The Dead Ends Are the Point

Most engineering teams document the fix. We started documenting what didn't work — and it turned out to be the more valuable half.

An incident log that records “the cause was X, the fix was Y” saves the next person the fix. An incident log that also records “we spent forty minutes convinced it was Z, and here is the command that ruled Z out” saves them the investigation. That is where the time actually goes.

The failure mode is quiet

A plausible-but-wrong hypothesis that nobody wrote down doesn't disappear. It sits there looking reasonable, and the next engineer — or the next AI agent — rediscovers it at full price.

This is what makes the cost so easy to miss. Nothing breaks. No alert fires. You simply pay for the same wrong turn repeatedly and never see the line item, because the time is distributed across different people on different days, each of whom experiences it as a normal amount of debugging.

The asymmetry is worth stating plainly: the fix is usually short and the investigation is usually long. We had been carefully preserving the short half.

One mandatory field

So the incident template has a required field: what did NOT work, and how it was ruled out.

Not a vague note that something was tried. The hypothesis as it was actually held, the command or observation that killed it, and what that command returned. An entry that says “we thought it was a permissions problem” is a shrug. An entry that says “we thought it was permissions; namei -l on the path showed the traversal was fine, so it wasn't” is a shortcut for whoever comes next.

The rest of the template exists for the same reason — context, the verbatim error text, the root cause, a reproducible command that verifies it, the fix that worked, and what it cost. Verbatim error text matters more than it looks: the next person's first move is almost always to paste the error into a search box, and a paraphrase doesn't match.

The harder half of the rule

Writing the log is the easy part. Reading it before you investigate is the part that takes discipline, and it is where the value actually is.

A session here once burned thirty minutes diagnosing an empty dependency graph. The cause was already written down — the repository had been hollowed out, files present but zero bytes — in a document that named that exact repository. The log was correct, current, and one search away. Nobody looked.

That incident changed the shape of the rule. It is not “write incidents down”. It is “write incidents down and consult them first”, and only the second clause pays. Writing logs nobody reads is half a system.

Why this matters more with agents in the loop

When the sessions doing the work are a mix of humans and AI agents, the re-derivation problem gets worse rather than better. An agent starts every session with no memory of the last one. It is fast, confident, and will happily spend its context re-deriving a conclusion that was reached and written down a fortnight ago.

A durable, searchable record of ruled-out hypotheses is one of the few things that reliably stops that. It turns institutional memory into something a fresh session can actually load — which is the whole point of writing any of it down.

What's the most expensive thing your team has debugged twice?