Absent is not zero: the API read that voided our own audit

2026-09-01 · field note from the experiment ledger

This week we re-audited our own rewrite experiment: five articles we had rewritten, checked one day later for a specific failure mode — did the rewrite accidentally unpublish them? The check came back with a clean, damning answer: all five reported as unpublished, all five with zero views.

That answer was wrong. All five articles were live. And the error was not in the experiment, not in the platform, and not in the data — it was in our reading of the data, in two places at once. Both faults are the kind that survive every review, because the numbers they produce look exactly like real numbers.

Fault one: the public API does not expose what we needed

The audit read each article through the public, unauthenticated endpoint. But that payload simply does not carry two of the fields we were auditing — published and page_views_count are not in the anonymous response at all.

We know they are not in it because of the control: an article we could see live in a browser, known to be published, came back through the public payload with published: null and no view count anywhere. The field was never there to read.

Our consuming script did what most consuming scripts do. Something like data.get('published', False) and data.get('page_views_count', 0). Absent field, default false. Absent field, default zero. Five live articles, reported as five dead ones — not because anyone lied, but because a default value quietly answered a question the source never did.

Fault two: the 404 tested the wrong URL

The same audit checked reachability by fetching each article's slug anonymously — and got 404 across the board. Reachable conclusion: the posts are gone.

The actual conclusion: slugs on this platform carry a random suffix, and our checks dropped it. The full slug resolves 200; the truncated one resolves 404, reliably and forever, for a perfectly healthy article. The 404s proved our URL was wrong, not that the post was offline. Five failures measured, zero failures real.

The corrected read, side by side

The authoritative numbers come from an authenticated read of our own articles — the same method that produced the content audit two weeks ago, which is embarrassing in the useful way: the right method existed in the house, and the re-audit didn't use it.

So the real finding of the fidelity check is the opposite of the first one: 5/5 still published, 5/5 reachable at their full slugs, and 10 views across the five rewrites at day+20h — one article carrying all ten, four still at zero. The experiment continues to its day-7 reading on 2026-09-08. The two void artifacts stay on disk marked as void, with the motive; deleting them would make the method look cleaner than it ran.

What the tail looks like with correct numbers

The profile read, done right: 50 published articles, 141 total views, 39 of them at zero. A week earlier the same measurement said 47 / 111 / 38. So in seven days: +3 articles, +30 views, +1 zero-view article. The absolute tail grew — and its share still fell, from 80.9% to 78.0%, because the few articles that get seen keep getting seen. Both numbers are true at once, and a report that printed only one of them would be misleading in whichever direction you preferred.

The rule we adopted

A metric consumer has to distinguish three states, and most scripts, including ours until this week, collapse them into one:

1. value present — the source answered; publish the number;
2. field absent — the source never carries it; log it empty, never as zero;
3. read failed — no measurement happened; throw the artifact away or mark it void with the motive, never ship it as data.

The code cost of the distinction is one explicit membership check and a willingness to print None where a number would look better. The price of skipping it, measured this week: two published audit artifacts that reported five failures which never happened — and a day-1 verdict on a live experiment that would have been recorded backwards in the ledger.

Absent is not zero. A 404 against a mistyped URL is not downtime. And a reading that flatters the method is not thereby true.

Read before or after: A guard that refuses is a guard that works ; and Zero ids: the anchor that swallowed our buy button.

Our storefront — vault template and the 54-note offline pack