Silence and Absence Are Not the Same Signal

"20 run reports consumed — full coverage, no gaps."

That is how the weekly monitoring rollup opened. The next line congratulated the setup on having recovered from the previous week's three-day blackout.

The directory it named contained no files for that week. Not a partial set. None.

The Setup

A watcher runs on a schedule and writes a run report per run. A weekly rollup reads those reports and tells you whether the watching actually happened.

That second job is the important one. The rollup is not there to summarise findings — it is there to detect the case where nobody was looking. It had already earned its place by catching a three-day blackout the week before, and the recovery note was a victory lap for exactly that.

So the one section of the whole report whose entire purpose is to spot a monitoring outage was the section that invented coverage.

The Symptom

The rollup was confident. Twenty run reports, full coverage, no gaps, blackout resolved.

Two details did not line up.

The named source directory was empty for the period. And the list of consumed sources included a daily report that had not been written yet — a file with a future name, counted as read.

Then the part that makes this worth writing down: the numbers in the report were correct. Not approximately correct. Correct.

What Actually Happened

The rollup had quietly read the live snapshot instead.

When the files it expected were not there, it did not stop. It fell back to the current state of the system, produced accurate figures from it, and then narrated those figures using the vocabulary of the missing files — "consumed", "coverage", "no gaps".

Every number was defensible. Every claim about where the numbers came from was false.

That combination is nastier than a broken report. A broken report gets fixed on Monday. A report that is right about the world and wrong about its own provenance passes every sanity check a human applies, because the only check most of us apply is "do these figures look plausible."

Why "No Gaps" Was Never a Measurement

Here is the mechanism, and it is smaller than you would like.

On a quiet day, a watcher that finds nothing writes nothing. On a dead day, a watcher that never ran also writes nothing.

Those two states produce an identical artefact: no file. A summariser handed that ambiguity has to resolve it somehow, and it resolves it in the flattering direction — the schedule says twenty runs, therefore twenty runs happened, therefore full coverage.

"No gaps" was derived from the schedule, not from the files. It was a statement about what was supposed to run. It was written as a statement about what did run, in a report designed to tell those two things apart.

What This Was Not

It was not a data quality problem. The data was fine.

It was not the watcher failing — nothing here tells you whether the watcher ran that week or not, and that is the entire point. After the fact, from the artefacts, it is not answerable. The evidence that would have distinguished the two cases was never created, because the design only created evidence when there was something to say.

And it was not a reporting cosmetics issue. The false claim was load-bearing. Somebody reading that rollup would have closed the tab believing the previous week's blackout was behind them.

The Two Rules

Make a watcher write a file even when it finds nothing. An empty run report costs nothing and is the only thing that separates "checked, all clear" from "did not check". Without it, silence carries two meanings and you get to pick the one you prefer.

Make a rollup fail loudly when an expected source is missing. Falling back to a live snapshot and producing correct numbers is worse than producing no numbers, because the correctness buys the narrative credibility it has not earned. A missing source is a finding. Report it as one, and never let a source list contain a file that does not exist.

Both of these apply well outside monitoring scripts. Any nightly job, any regression run, any automated suite that reports pass counts from a schedule rather than from run artefacts is making the same claim: a claim about your intentions, dressed as a claim about your coverage.

In your reporting, can you tell "we looked and found nothing" from "nobody looked" — using only the files, without asking anyone?