When your docs drift from your product

There is a particular kind of wrong that only happens to documentation. It is not the obvious kind, where a page describes a button that no longer exists and a reader files a bug within the hour. It is the quiet kind: a number that was carefully verified once, copied into the places that needed it, and then left behind when the product moved. Every copy agrees with every other copy, so nothing looks inconsistent. The site reads as confidently as it ever did. And the only thing in the world that disagrees is the source code, which nobody consults while editing prose.

We ran a full coherence audit across a portfolio of small, free web apps and their shared marketing hub, and that is exactly the shape of what we found. Not sloppy writing — the writing was fine. Not missing pages — coverage was broad. What we found was a set of facts about the products that had been true, had been faithfully replicated, and had silently expired. This is a field guide to that failure mode: how it forms, how to measure it cheaply, how to fix it so the same fact cannot rot in twenty places at once, and what to tell readers when you correct something they may already have read.

The instinctive response to a wrong number on a website is to fix the number. That is a fine response to the symptom and a useless response to the cause. The cause is that the number had no home. It existed as a literal string in one file, and when a second surface needed it, someone typed it again. When a third needed it, someone copied from the second. Within a quarter the fact lived in a dozen places, each one a separate opportunity to go stale, with no relationship between them that any tool could see.

Framed that way, the problem stops being about writing discipline and starts being about data modelling, which is a much more tractable subject. A fact with one canonical definition and many references can be updated once. A fact that has been duplicated into free text can only be updated by a human who remembers every place it was pasted, which is to say it cannot reliably be updated at all. Every documentation system that stays accurate over years has solved this in some form, whether through variables, includes, generated pages, or a build step that reads the product and writes the prose.

The corollary is uncomfortable but freeing: if your copy has drifted, your process was going to produce drift eventually no matter who was writing. Blaming the last person to touch the file is both unfair and unhelpful, because the next person will make the same mistake for the same structural reason. The fix has to change the shape of the system, not the diligence of the individual.

Auditing documentation by reading it is slow, boring, and surprisingly ineffective — familiar prose slides past the eye, and the claims most likely to be stale are exactly the ones that look most authoritative. A far better first pass is mechanical: decide which facts your copy leans on, then search for each of them as a literal string across the entire content tree.

Numbers are the ideal target because they are unambiguous, easy to grep, and the single most common thing to go stale. Counts of supported formats, numbers of tools, sizes of a library, lengths of a feature list, names in a roster of integrations — these are the claims that sell the product and the claims that expire first. Feature names are worth searching too, along with status words like "beta", "coming soon", and "planned", which have a habit of outliving the state they described by many months.

The output of that search is not a to-do list, it is a map. For each fact you now know exactly how many surfaces assert it and where they are. Then you do the part that actually requires judgement: open the file in the product that produces the real value and read it. Not the product's marketing page — the registry, the catalog, the type definition, the thing the application itself consults at runtime. That file is the only authority. Everything else is a claim about it.

The concrete findings are more instructive than the method, so here they are without softening. A file converter that the hub described as publishing 1,022 conversion routes had grown to 1,636 working routes across 209 browser-ready input formats, from a registry documenting 243 formats, with 66 same-format compressors alongside them. A music visualizer described everywhere as having nine engines with six modes each had shipped a tenth engine, taking it to ten engines and sixty modes. A library of generated test files listed at 2,428 files was at 2,558 across seventeen categories and 221 distinct extensions, with its visual template library up from 480 to 504 across 42 categories. An AI usage dashboard documented as supporting six import sources supported seven.

None of those gaps came from carelessness. Every one of them came from the same mechanism: the product shipped, the number changed in the code, and the twenty copies of the old number in the prose had no idea. The largest single drift — the visualizer engine count — appeared in over a hundred places across more than two dozen source files, because it had been faithfully repeated in article bodies, card highlights, metadata descriptions, FAQ answers, link descriptions, and image alt text over several months of writing.

The most interesting finding was not a wrong number at all. It was an absence. One app had grown an entire second half — a careers and assessments layer with hundreds of occupation profiles, hundreds of aptitude practice suites, thousands of authored questions, a puzzle catalog, and a full authored programming course — and a search across every marketing surface for the words that describe it returned zero hits. Drift does not only make your copy wrong. It makes it incomplete in ways nobody notices, because nothing on the page looks broken.

A wrong number is embarrassing and easy to fix. Missing coverage is neither. When a product grows a capability that no page describes, there is no error message, no failing test, and no reader complaint — the people who would have wanted that capability simply never learned it existed, and they are not in a position to tell you what they did not find. The feedback loop is completely silent.

That silence is why absence needs a different detection method than wrongness. Grepping for a stale number works because you know what to search for. Grepping for a missing feature does not work, because the search term is precisely the thing that is not there. The only reliable approach is to invert the direction: start from the product, enumerate what it can do, and check each capability against the copy. A route list, a navigation manifest, a public sitemap, or a catalog of content types all make good enumeration sources because they are generated from the thing itself.

Doing that inversion once a quarter is cheap and catches the expensive gaps. It also produces a useful by-product: a list of capabilities ranked by how much documentation they have, which is a decent proxy for where your next piece of writing will do the most good.

The structural fix is unglamorous and works: define every statistic exactly once, in a module whose only job is to hold statistics, and have every surface reference it rather than restate it. In a content-as-code site this is a plain source file exporting typed constants. In a CMS it is a set of global fields. In a generated docs site it is a data file the templates read. The technology matters far less than the rule, which is that a digit describing your product may appear in exactly one file.

Two details make the difference between a module that helps and one that becomes another thing to maintain. The first is that every value should carry a comment naming the upstream file it was verified against, so re-checking is a single file read rather than an investigation. The second is that composite figures should be computed from their parts rather than typed separately: if you publish both "ten engines with six modes each" and "sixty modes", the second number must be the product of the first two in code, or the day someone adds an engine you will publish a roster and a total that contradict each other.

It is worth being explicit about what should not be centralised, because over-applying the pattern causes its own damage. Anything that is a record of a moment must stay frozen. A changelog entry describing what shipped on a date is not a claim about today, and wiring it to a live value would silently rewrite history every time the product grew. The same is true of a dated announcement post: it should say what was true when it was published, and any correction should be visible as a correction rather than a seamless retcon.

That distinction — record versus claim — turns out to be the most useful editorial question in the whole exercise, and it is worth applying deliberately to every page rather than intuitively. A claim is a statement about the present: this is what the product does, these are the formats it accepts, this is how many there are. A claim that has gone stale is simply false and should be corrected. A record is a statement about a moment: on this date we shipped this, and the catalog stood at this size. A record that no longer matches the present is not false at all, and rewriting it destroys information.

The awkward cases are the pages that mix both, which in practice is most announcement posts. A release write-up naturally contains a historical narrative — this went from seventy files to four hundred — and a set of present-tense assertions about where things now stand. The correct treatment is to leave the narrative alone and to date-anchor the assertions, so a sentence that used to read "the catalog is now X" reads "at the time of this release the catalog stood at X", followed by a current figure with its own date. The reader gets both the history and the truth, and neither one is pretending to be the other.

FAQ blocks deserve particular attention because they are almost always claims, they are almost always written in the present tense, and on many sites they are also emitted as structured data. A stale answer inside a FAQ block is therefore wrong twice: once to the human reading the page and once to the machine that indexed it. When auditing, treat the FAQ as the highest-priority surface on any page.

Silent correction is tempting and slightly dishonest. Someone read that page last month, took a number from it, and possibly repeated it somewhere. Quietly swapping the digits leaves them stranded with no way to know their copy is now wrong, and it deprives every other reader of a useful signal about how the material is maintained.

A small, specific review notice fixes that at almost no cost. Not a bare timestamp — a bare timestamp says nothing and is easily faked by a rebuild — but a sentence naming what was re-checked and what changed. "Re-checked against the format registry on this date; the route count was updated from 1,022 to 1,636" is a genuinely useful sentence. It tells a returning reader exactly whether their takeaway survived, and it tells a first-time reader that someone is watching the page.

Two rules keep the notice trustworthy. Apply it only to pages whose content actually changed, because a review badge on an untouched page is a claim about work that did not happen. And keep the machine-readable modification date in step with it, so the structured data reports a real revision rather than the date of the last deploy. A freshness signal is only worth having while it is still true.

Everything above is judgement work, and judgement does not scale on its own. The complement is a set of mechanical gates that fail loudly and cost nothing to run. If your content lives as typed code, the compiler is already doing part of the job: a registry keyed by a union of valid slugs will reject a reference to a page that does not exist, which quietly eliminates a whole class of broken internal link before anyone reviews anything.

Beyond types, the checks worth automating are the ones a human cannot reliably do at volume. Every internal link resolves to a real route. Every image path exists on disk at its declared dimensions. Every article meets a minimum length, so a thin page cannot slip out. Every published entry belongs to a category rather than falling through to a default bucket. Every share image has a matching raster twin. None of these are clever, and each one catches a failure that would otherwise reach a reader.

What automation cannot do is decide whether a paragraph is still true. That is the permanent human job, and it is the reason the audit and the gates are complements rather than alternatives. The gates hold the structural floor so that human attention can be spent entirely on meaning, which is the only place it produces real value.

There is a practical argument for all of this that has nothing to do with craft. Search guidance has moved steadily toward rewarding content that demonstrates first-hand knowledge and accuracy, and away from content that merely covers a topic. A page that confidently states a number a reader can check and find wrong is exactly the pattern that guidance is designed to demote. The cost of drift is no longer only reputational.

The same is increasingly true for the machine readers. Assistants and answer engines lift specific claims out of pages and repeat them in contexts where the source is invisible and the reader has no easy way to verify. A stale figure on your site does not stay on your site — it propagates, unattributed, into answers you will never see. Whatever you think of that, it raises the cost of being casually wrong about your own product.

None of this argues for writing less or hedging more. It argues for making the claims you can stand behind, keeping them in one place so they can be maintained, and dating the ones that are historical. That combination lets you write confidently and specifically, which is what makes technical content worth reading in the first place.

A quarterly full audit sounds responsible and tends not to happen, because it is large, undated, and owned by nobody. Two smaller habits work better. The first is a release-time rule: whenever a change alters a number the marketing copy quotes, the same change updates the canonical definition. Because the definition lives in one file, that is a one-line edit rather than a scavenger hunt, which is precisely why the centralisation is worth doing.

The second is a short, scheduled pass with a fixed scope: verify every statistic against its named upstream file, check the newest capability for coverage, and stop. It does not read every page, it does not rewrite anything for style, and it finishes. A narrow pass that reliably happens is worth more than a comprehensive one that gets postponed twice and then abandoned.

It also helps to give the audit an artifact. A short written record of what was checked, what was wrong, and what was changed makes the next pass much faster, and it makes the work visible to anyone who has to decide whether it is worth continuing to fund. Invisible maintenance is the first thing cut.

For the record, the remediation in our case had three parts, and they map onto the three problems above. Every statistic the hub quotes about an app now lives in one module, annotated with the sibling-repository file it was verified against, with composite figures computed from their parts rather than typed twice. Every article whose claims were corrected carries a dated review notice naming what changed, and reports that date as its modification date to search engines rather than repeating its publication date. And the capability that had no coverage at all got written about properly, because a documentation system that only corrects what already exists will never fix an absence.

The part that took longest was not any of that. It was reading each of the corrected paragraphs and deciding whether the surrounding argument still held once the number changed. Sometimes it did not: a sentence built on "the last two engines carry the most spectacle" needs restructuring, not a find-and-replace, when a tenth engine arrives that belongs in neither group. That is the work no tool does for you, and it is a good reason to treat a documentation audit as an editorial exercise with mechanical assistance rather than a mechanical exercise with editorial cleanup.

Documentation drifts because facts get duplicated into prose where nothing can maintain them. Find the drift by grepping for the claims rather than reading the pages, and find the absences by enumerating the product rather than searching the copy. Fix the cause by giving every fact exactly one definition and referencing it everywhere else, while leaving genuine historical records frozen. Tell readers when you corrected something and what it was. Automate the structural checks so human attention goes to meaning. Then keep a narrow, scheduled pass that you will actually run.

The payoff is not tidiness. It is that specificity becomes safe again. Once a number can only be wrong in one place, you can put real figures in your writing instead of retreating to vague language to avoid being caught out — and specific, checkable, first-hand writing is the only kind that earns a reader's trust in the first place.