Three Weeks, Three Failure Classes
Step discount failed three weeks running and the failure class changed every week, because each retest was scoped to whatever the previous week's bugs had named.
Same feature. Three weeks. Three rounds of defects.
And not once did the same kind of failure come back.
What Each Week Found
Week one: six single-surface failures. Step discount, applied on its own, wrong in six separate places. Six bugs, six fixes, six retests. Clean.
Week two: a combination failure. Step discount together with a service discount and a security add-on. Each of those had passed alone. Together they produced the wrong price in the billing system.
Week three: a state-transition failure. The discount duplicated after a product was added to the cart and then removed.
Read that sequence twice, because the pattern in it is worth more than any of the three bugs.
Week one was about a feature in isolation. Week two was about the feature meeting other features. Week three was about the feature meeting a sequence of user actions. Those are three different classes of failure, and the feature moved through all of them without repeating itself once.
Why It Kept Moving
Because every retest was scoped to the surfaces the previous week's bugs had named.
That is a completely reasonable-sounding scope. Six bugs were raised, six fixes landed, and the obvious next step is to prove the six are fixed. So the week two run tested the six surfaces from week one. The week three run tested the combination from week two.
Each round proved exactly what it set out to prove. Each round was also blind, by construction, to any class of failure that had not yet been discovered.
Re-running a bug list re-tests what already failed. It cannot find anything else, because the bug list is a record of the past, and a scope derived from it inherits the shape of whatever you happened to find first.
Week one found single-surface bugs, so week two looked at single surfaces. That is why week two's real defect was a combination — combinations were the nearest thing nobody was looking at.
The Loop This Creates
There is something quietly corrosive about this pattern beyond the defects themselves.
Three weeks in, the feature has a reputation. People start saying it is unstable, or badly built, or that the developer keeps breaking it. None of that is supported by the evidence. What the evidence actually shows is a test set that never grew — it moved, week by week, following the defects instead of anticipating them.
And each week produced a green result at the end. Genuinely green, for the scope it ran. Which is why the next week's failure came as a surprise every single time.
What It Was Not
It was not a regression. Nothing that had been fixed came back broken.
It was not a fix-quality problem either. Every fix held.
And it was not bad luck landing three times in a row. The third round was as predictable as the second, given how the scope was being chosen — the only thing you could not predict in advance was which unexplored class would surface first.
Re-derive, Do Not Re-run
After a defect cluster, the useful move is not to retest the bugs. It is to re-derive the test set from scratch, from a matrix, and let the bugs be evidence about where the previous derivation was thin rather than the input to the next one.
The matrix has four axes, and this feature demonstrated three of them the hard way:
- The feature crossed with other features, and with add-ons. This is the axis that produced week two. You do not need every combination — a pairwise table over the discount types and the add-ons would have generated that exact case at design-of-tests time, before a single execution, and it is an afternoon's work to build.
- The feature crossed with cart mutations. Add, remove, re-add, change quantity. This is the axis that produced week three, and it is the one most consistently missing from suites, because a test plan describes what the record should end up containing rather than what was done to it on the way.
Build that matrix once and the three weeks collapse into one derivation. More usefully, the shape of the matrix tells you what you are not covering, which a bug list can never do — a bug list only ever tells you where you have already been.
When your feature fails twice, do you widen the scope, or do you re-run the last one with the fixes in?