Audit every tool, not just the broken one: the "all-tools" doctrine behind our refactors

A bug report names one tool. The background remover's Best Quality model fails, or the video tool stops producing output, or some specific feature misbehaves. The natural and tempting response is to fix that tool — find the problem in the named place, correct it, close the ticket. The doctrine we follow instead is to treat the report as a sample rather than the whole problem, and to audit every tool that shares the relevant pattern, not just the one that happened to surface the failure. This post is about why a bug in one place is almost always a pattern in several, and why fixing the pattern across the whole suite — even the parts nobody reported — is what actually ends the bug instead of postponing it.

The doctrine has a name in how we work — the all-tools approach — and it came directly from experience rather than theory. The pattern it guards against is whack-a-mole: the same underlying bug surfacing in tool after tool over time, each one fixed in isolation, each fix giving the false impression that the problem is handled until the next tool surfaces it again. The all-tools doctrine is the decision to stop playing that game by fixing the pattern everywhere the first time it is understood.

The reason the single-tool fix is a trap is that tools in a suite share patterns. They are built by the same people, following the same conventions, often by adapting one tool's code into the next, which means a flaw in a shared pattern propagates to everywhere that pattern was used. When a bug is rooted in such a pattern — a lifecycle that must be managed a certain way, a piece of state that must be handled consistently, an assumption that must hold — the place it gets reported is just the first place a user happened to trigger it, not the only place it exists. The same flaw is sitting, unreported, in every other tool that followed the same pattern, waiting for a user to find it there too.

This was concrete in the background remover. A model-session lifecycle bug surfaced in one place, but the pattern of managing that lifecycle by hand existed everywhere a model ran. Several distinct queue implementations had drifted apart over time while sharing the same underlying responsibilities, so a bug in how one of them handled cancellation or worker cleanup was very likely lurking in the others. Fixing only the reported queue would have left the same bug live in the rest, guaranteeing that the same issue would resurface tool by tool — the exact whack-a-mole the doctrine exists to prevent. Seeing the report as one instance of a pattern, rather than as the whole bug, is what made the difference between ending the problem and merely deferring it.

Acting on the all-tools doctrine changes what a fix is. Instead of correcting the behavior in the reported tool, the fix becomes: identify the pattern the bug is rooted in, find every place that pattern occurs, and bring all of them to a correct, consistent implementation at once. For the queue bug, that meant not patching the one queue that surfaced the failure but reconciling all of the divergent queue implementations to a single canonical model, so that the correct handling of cancellation, errors, and worker lifecycle existed identically everywhere rather than being fixed in one place and left broken in the others. The fix addressed the pattern across the whole suite, which is more work in the moment and dramatically less work over time, because it ends the recurrence instead of treating one symptom of it.

This is why the doctrine pairs so naturally with rebuilding when patches compound and with research-first investigation. The research sprint is often what reveals that a reported bug is actually a pattern in many places — you go looking for every occurrence and find more than you expected. The rebuild is often the right shape of an all-tools fix, because bringing many divergent implementations to one consistent model is exactly what a rebuild does. And the all-tools audit is the discipline that ensures the rebuild covers everything the pattern touched rather than stopping at the reported case. The three reinforce each other: investigate to find the full extent of the pattern, rebuild to a consistent model, and audit to confirm the fix reached every instance.

It helps to name the specific failure mode the all-tools doctrine exists to prevent, because it is so common and so deceptively comfortable: whack-a-mole, where the same underlying bug surfaces in one tool after another over time, each instance fixed in isolation, each fix giving a false sense that the problem is now handled. The comfort of whack-a-mole is that each individual fix feels like progress — a bug was reported, a bug was fixed, the ticket closed — so the team feels productive even as the same root problem keeps generating new reports. The discomfort only accumulates gradually, as it becomes clear that fixing these bugs never seems to reduce their rate, because the root pattern keeps spawning fresh instances faster than they are individually swatted.

Recognizing whack-a-mole as a distinct failure mode is what motivates breaking out of it. The trap is that each instance, viewed alone, genuinely looks like a separate bug deserving a separate fix, so the pattern is invisible if you only ever look at one report at a time. It is only by stepping back and noticing that these bugs rhyme — that they are the same flaw wearing different tool-specific clothing — that the underlying pattern becomes visible and the whack-a-mole becomes obviously futile. The all-tools doctrine is the decision, made once that recognition lands, to stop swatting instances and start eliminating the pattern, which is the only move that actually reduces the rate of these bugs rather than maintaining a steady stream of them to fix forever.

The structural reason a bug in one tool is usually a pattern in many comes down to how a suite of tools is actually built: by the same people, following the same conventions, frequently by adapting one tool's code as the starting point for the next. This shared lineage means the tools are not independent — they carry common patterns, and a flaw in a shared pattern is inherited everywhere that pattern was used. When a bug is rooted in such a pattern, the tool where it was reported is simply the first place a user happened to exercise the flaw, not the only place it lives. The same flawed pattern sits, unexercised but present, in every sibling tool built the same way.

This is why the place a bug is reported tells you almost nothing about where the bug actually exists. The report reflects where a user went, not where the flaw is, and the flaw is wherever the pattern is. In the background remover's case, a model-session lifecycle bug surfaced in one tool, but the pattern of managing that lifecycle by hand existed everywhere a model ran, and the divergent queue implementations shared the same underlying responsibilities — so the bug reported in one queue was almost certainly present in the others. Understanding that shared construction propagates shared flaws is what reframes a single bug report from "this tool is broken" to "this pattern is broken, and here is one place it showed." That reframe is the entire basis of the doctrine, because it is what tells you to look beyond the reported tool to every tool that shares the pattern.

The central rebuttal to the "all-tools is too much work" objection is that the single-tool fix does not avoid the cost of fixing the other instances — it defers that cost and inflates it. The other instances of the pattern do not cease to exist because you only fixed the reported one; they remain, live, waiting for users to trigger them, at which point each becomes its own bug report, its own diagnosis from scratch, its own fix, its own regression risk. The work of fixing them was not saved by the single-tool fix; it was postponed and scattered across future incidents, with the diagnosis cost paid again each time because the connection to the original bug is not obvious when it resurfaces months later in a different tool.

Worse, the deferred fixes tend to diverge, because they are done separately at different times by someone who may not remember exactly how the original was fixed — so instead of one consistent fix applied everywhere, you accumulate several slightly different fixes for the same flaw, which is its own source of future inconsistency and bugs. Paying once, upfront, to fix the pattern across every instance is therefore not extra work compared to the single-tool fix; it is the same work, done once and consistently, instead of many times and divergently. The single-tool fix looks cheaper only because its true cost is hidden in the future. Accounting for that deferred, inflated, scattered cost is what makes the all-tools doctrine obviously economical rather than obviously expensive, despite looking slower in the moment of any single fix.

It is useful to see the all-tools doctrine not just as a bug-fixing strategy but as a guardrail — a standing rule that prevents the partial fix from being treated as complete. Without the doctrine, the default definition of "fixed" is "the reported instance no longer reproduces," which is a definition that licenses whack-a-mole by declaring victory at the first instance. The doctrine changes the definition of done for a pattern-rooted bug: it is not fixed until every instance of the pattern is fixed, which means a fix that touches only the reported tool is incomplete by definition rather than complete by default. Reframing the standard of completeness is what stops partial fixes from quietly shipping as finished work.

This guardrail function is why the doctrine has to be an explicit, standing principle rather than a case-by-case judgment, because under time pressure the case-by-case version always drifts toward declaring the reported instance fixed and moving on. A pre-committed rule — when a bug is rooted in a pattern, the fix covers every instance of the pattern — holds against that pressure precisely because it was decided in advance, the same way the reversibility-based guardrails for agent autonomy hold because they were drawn deliberately rather than in the moment. The doctrine is a guardrail in the same family: a deliberate, standing line that protects a quality standard against the in-the-moment temptation to do the easier, partial thing. Making "fix the whole pattern" the non-negotiable definition of done for pattern bugs is what operationalizes the doctrine rather than leaving it as good intentions.

Intellectual honesty requires noting that not every bug is a pattern in disguise — some are genuinely local, a one-off mistake confined to a single place with no shared root, and applying the all-tools doctrine to those would be wasted effort. A typo in one function, a logic error specific to one feature's unique requirements, a mistake that does not stem from any pattern the other tools share — these are correctly fixed where they are reported, because there is no pattern to audit across the suite. The doctrine is not "every bug fix must touch every tool"; it is "a bug rooted in a shared pattern must be fixed across every instance of that pattern," which only applies when there is a shared pattern.

The skill, then, is distinguishing a pattern-rooted bug from a genuinely local one, because the right response differs completely. The tell of a pattern-rooted bug is that it stems from something the tools share — a common lifecycle, a common piece of state, a common convention — so that the same flaw could exist anywhere that shared thing is used. The tell of a local bug is that it arises from something specific to the one tool, with no shared root that would propagate it. Asking, when a bug is found, "is this rooted in something the other tools also do, or is it specific to this one" is what determines whether to audit the suite or just fix the spot. Misjudging this in either direction wastes effort — auditing for a local bug, or spot-fixing a pattern bug — so the discipline includes the judgment of which kind of bug you are looking at, not just the response once you know. The doctrine is precise: fix the pattern when there is a pattern, fix the spot when there is not.

The all-tools doctrine is usually framed as a bug-fixing strategy, but its largest long-term payoff is a consistency dividend that improves the codebase well beyond the specific bugs it eliminates. Every time a pattern is fixed across every instance rather than in one place, the suite ends up with that pattern implemented identically everywhere, which is a step toward the internal consistency that makes a codebase easy to reason about and change. A codebase where the same thing is done the same way throughout is dramatically more maintainable than one where each tool does it slightly differently, because understanding one instance means understanding all of them, and changing the pattern means changing it in one consistent place.

This dividend compounds in the opposite direction from the fragmentation that single-tool fixes produce. Each isolated fix tends to add a slightly different implementation of the same thing, pushing the codebase toward the divergent state where the same pattern exists in many subtly inconsistent forms — which is itself a breeding ground for the next hidden bug. Each all-tools fix pulls in the other direction, toward one consistent implementation, leaving the codebase a little more coherent than before. Over many such fixes, the difference between the two trajectories is enormous: one leads to a tangle of divergent implementations, the other to a clean, uniform suite. The all-tools doctrine is therefore a force for maintainability as much as a bug-fixing rule, and the consistency it accumulates is the deeper reason it is worth the upfront work, beyond the immediate bugs it ends.

The objection to the all-tools doctrine is obvious: fixing every instance of a pattern is more work than fixing the one that was reported, and when you are trying to move fast, doing extra work on tools that are not currently broken feels like a poor use of time. The answer is that the extra work is not extra; it is the same work, done once instead of many times. The single-tool fix does not avoid the work of fixing the other instances — it defers it, and adds to it, because each deferred instance will eventually surface as its own bug report, its own diagnosis, its own fix, its own regression risk, with the additional cost that the fixes will likely diverge since they were done separately at different times. Paying once to fix the pattern everywhere is cheaper than paying repeatedly to fix each instance as it surfaces.

There is also a quality dimension beyond cost. A suite where the same pattern is implemented consistently everywhere is easier to reason about, easier to change, and less likely to harbor the next hidden bug, because there is one way things are done rather than several slightly different ways. The all-tools doctrine is therefore not just a bug-fixing strategy but a force toward the kind of internal consistency that keeps a growing codebase maintainable. Every time a bug report is treated as a sample of a pattern and the pattern is fixed across the suite, the codebase gets a little more coherent rather than a little more fragmented. That compounding coherence is the real payoff, and it is why the doctrine is worth the upfront work that makes it look slower in the moment and is faster over any horizon that matters. The companion case study traces the specific bug that taught us this, and the rebuild retrospective shows the doctrine applied at full scale.