Vary the Base Order, Not the Target Product

The product was in the list. And it was not in the list.

Both were true. Which one you saw depended on what the customer had bought months earlier.

A regression run in SIT found an add-on missing from the service-change product list whenever one specific higher-tier plan sat in the original activated order. Highest priority, real defect, and completely invisible to any test that starts from a different base.

Visibility Is Not a Property of the Product

This is the mental model correction, and it is worth stating plainly.

When you test a product in a join journey, its availability is close to a property of the product itself. It is in the catalogue, it is eligible, it appears. One test, one answer.

In a change journey, that stops being true. What appears in the list is computed against the subscription the customer already holds — compatibility rules, exclusions, what the base order contains. The product is not visible or invisible. It is visible given something.

So the answer to "is this product available in service change?" is not yes or no. It is a function, and you have evaluated it at exactly one point.

What One Vanilla Base Order Proves

Almost nothing, and it feels like a lot.

Most change-journey testing starts from whatever activated order the tester could get hold of. Usually the simplest one, because it is the fastest to produce. The product appears, the case passes, the coverage report gains a row.

That row now says the product is available in service change. What it actually says is that the product is available in service change from one starting state out of however many the compatibility matrix distinguishes.

The gap between those two statements is where the defect lived. Not in the product configuration, not in the journey logic — in the space of starting states nobody enumerated.

The test matrix here has a dimension that rarely gets written down: starting subscription state. Every compatibility rule needs at least one base order where the product should appear and one where it should not. Two cases per rule. Anything less is not testing the rule, it is sampling it.

The Same Blind Spot, One Layer Down

The starting state matters on the way out as well.

Removing a static IP product from an order drops the Static IP Subscription Id from the payload sent to the provisioning system — but only when the static IP is deprovisioned in the same modify transaction as the VoIP product. Remove either one on its own and the payload is correct. Remove both together and the field disappears.

That is not a variant of a tested scenario. It is a scenario nobody ran.

Remove paths are systematically under-tested compared to add paths, for the same reason vanilla base orders dominate: adding is what the demo shows, adding is what the acceptance criteria describe, and adding is what the tester does first. Removal gets one case, usually one product, usually the simplest one.

And a combination removal builds a different payload than a single removal. Different fields, different assembly, different code path. Testing one-product removal gives you a green result and tells you nothing about the two-product case sitting next to it.

The Constraint You Assumed Was Behind You

A third version of the same error, from the other direction.

Two tiers of the same add-on family are mutually exclusive. To add the higher tier when the standard one is active, you must explicitly remove the active product first, in the same session. Attempt the add without the remove and the journey blocks you with an error.

Juniors assume the modify journey behaves like a fresh cart. It does not. The exclusivity rule from the join journey carries straight into in-life changes, and the remove-then-add sequence is a prerequisite that appears in no happy-path walkthrough anywhere.

So the tester hits the error, assumes a defect, and files it. Or worse, assumes their environment is broken and spends two hours proving it is not.

What to Actually Change

Three habits, and they cost less than they sound like.

• Make the base order an explicit column in your test conditions. Not a precondition sentence buried in setup — a dimension with values, so that a reviewer can see at a glance how many starting states the suite covers and which compatibility rules have only one.
• Give combination removal its own case, and assert on the payload. Wherever two products can be deprovisioned in a single transaction, run that combination and read what actually left the system. The UI will look identical in both cases; the dropped field only exists in the callout.
• Re-validate join-journey constraints inside the change journey. Exclusivity, dependency and eligibility rules do not stop applying in-life, and the sequence needed to satisfy them has to be written into the case rather than left to the tester to discover.

Why This Dimension Gets Dropped

Base orders are expensive.

Every distinct starting state needs a full activated order, which means running the whole join journey, waiting for orchestration, and holding onto the result before somebody's data refresh takes it away. A matrix with eight starting states is eight of those.

So the matrix collapses to whatever base orders exist in the environment on the day, and the test conditions get written to fit them. Nobody decides to skip the dimension. It gets skipped by arithmetic.

Which is exactly why it belongs in the test design, visibly, before anyone starts building data. A base order you decided not to cover is a documented risk. A base order that never occurred to you is a defect waiting for regression to find — one release later, at Highest priority, in somebody else's window.

How many different starting subscription states does your change-journey suite actually cover — and do you know the number, or are you about to guess?