Your Quality Metrics Need a Diff, Not a Snapshot

A defect was closed. On the same day it disappeared from the issue links of both stories it had been raised against.

Only the daily diff caught it. A snapshot taken the following morning would have shown two stories, each with one fewer defect, and nothing anywhere indicating that a defect had ever been there.

The Link Graph Is Mutable and Unversioned

This is the mechanism, and it applies to every tracker I have worked in.

The relationship between a defect and a story is a record like any other. It can be created, and it can be deleted. When it is deleted, it does not go to an archive that your queries can reach — it goes out of the graph. The relationship stops having existed.

So any query you run reads the graph as it is at the moment you run it, not as it was when the work happened. Every "defects per story" figure, every leakage calculation, every first-time-pass rate built on bug-to-story links is computing over a structure that anyone with edit rights can rewrite, retroactively, with no trace in the thing you are measuring.

A snapshot cannot show a removal. That is not a limitation of your query. It is arithmetic: a removal is a difference between two states, and a snapshot is one state.

Nobody Is Doing This Maliciously

Worth saying early, because the instinct on reading the opening is that someone was covering something up.

Links get removed for entirely ordinary reasons. A defect was linked to the wrong story and someone tidied it. A duplicate was cleaned up. Somebody re-pointed a bug at the story they thought owned it more properly. A bulk edit did something nobody inspected.

The intent does not matter to the number. Whatever the reason, the effect on your metrics is identical: history was edited, and the edit is invisible to everything downstream of it.

That is why this needs a mechanism rather than a rule about behaviour. You cannot police link hygiene across a programme. You can record what the graph looked like yesterday.

The Second Thing a Diff Sees

Once you are taking daily snapshots for one reason, they answer a sharper question for free.

A story moved from Testing In Progress to Ready For Deployment while two of its own linked bugs were still sitting in Developing. One of those bugs prices an £88 product at £8 when the agent removes the promotion but keeps the custom discount. Two other stories in the same sprint showed exactly the same shape.

Nothing in the workflow stopped any of them. The tracker treats the story's status category as green while its defects are still yellow, and there is no rule anywhere that says a story cannot travel downstream with its known bugs riding along.

The consequence lands squarely on QA. Those defects were found on time, by testing, during testing. When they resurface in system integration testing or UAT, they get counted as escapes — recorded against the team that had already found them, because the record shows a defect appearing in a later phase and nothing shows that it was raised, known and shipped anyway.

The Rule

Treat "story done" and "defects closed" as two separate states, and measure the gap between them over time rather than at a point.

Both of the findings above come from the same wrong assumption: that the current state of the tracker is the record. It is not. It is the latest revision of the record, and revisions are cheap.

Building It

• Snapshot daily, not weekly. Story key, story status, linked defect keys, each defect's status, timestamp. That is the whole payload, and a week's granularity misses a link that was created and removed inside the same sprint — which is precisely the case that matters.
• Alert on the delta, not the state. A link that existed yesterday and does not today. A defect that closed on the same day it was unlinked. A story whose defect count fell without any defect closing. Each of those is one comparison, and each is invisible to any query run afterwards.
• Run one standing query every day: stories in a forward status whose linked defects are still open. This is the cheapest escape predictor a test lead has. It costs nothing to build, it names specific stories before they leave the building, and it produces a list short enough to walk through in a stand-up.
• Keep the snapshots outside the tracker. A history stored in the system whose history you do not trust is not a history. A flat file per day is enough, and being able to answer "what did this story look like on the twelfth" is worth more than any dashboard you could build on top of it.

Why This Escapes Almost Everyone

Because metrics get consulted at the end of the period, and the graph gets edited during it.

Nobody opens the quality numbers on a Tuesday afternoon in week two of a sprint. They open them at the retrospective, at the release review, at the point where somebody asks how testing performed. By then every edit made in the intervening three weeks has already been absorbed into the picture, silently and without contradiction, and the picture is perfectly coherent.

There is no error state to notice. A story with three linked defects and a story that had four and now has three look identical. Nothing is missing from the screen, because "missing" is only visible against a previous version, and the tool is not offering you one.

Pick a story that shipped last month. Can you prove how many defects were linked to it while it was in test?