Five of the Six Were Configuration

We were building the case for a different test management tool.

Not out of fashion. Out of daily friction — the kind where six separate things you need to do are each slightly harder than they should be, and the accumulated weight of that makes the incumbent tool feel like the problem.

Then somebody ran a desk-check instead. Six needs, written down, one at a time, against what we already owned.

Five of the six were configuration.

The Reflex, and Why It Is So Convincing

"This tool is bad, let us buy a better one" is a genuinely attractive proposition, and not because anybody is lazy.

It converts a diffuse, ongoing irritation into a single decision with a budget line and an end date. It gives the frustration a name. And it comes with a demo where every one of your six needs is shown working, because that is what demos are for.

What it hides is the comparison you never actually ran. You know your tool's failures intimately, because you have hit them personally, on a deadline. You know the other tool's capabilities from marketing. That is not a comparison. It is a memory of pain set against a promise.

Most tool pain is not an architectural limit. It is a configuration gap plus a knowledge gap, and both of those travel with you to the new tool.

Write the Needs Before You Write the Case

The desk-check starts with a list that has nothing to do with either product.

Six needs, phrased as jobs rather than features:

• Give a test case a lifecycle status. Draft, ready, approved, deprecated — so that a suite can distinguish a case that is finished from one that somebody is still writing.
• Version a test case. When a requirement changes and the case is rewritten, keep the earlier version rather than overwriting the only record of what was previously agreed.
• Separate execution from the case itself. A run is an event with a date, an environment and a result; the case is the definition. Conflating them means you lose history every time you re-run.
• Assign an execution to a person. Regression is divided among people, and the division needs to live in the tool rather than in a spreadsheet or a stand-up.
• Manage suites. Group, re-group and maintain sets of cases — a smoke set, a release set, a per-feature set — without duplicating the cases themselves.
• Report on peer review. Show what has been reviewed by a second tester and what has not, because unreviewed cases are exactly where the coverage claims come apart.

That list is the artefact. It is more valuable than the tool decision, because it is what you actually need — and unlike a feature comparison, it does not expire when the market changes.

Then Check It Against What You Own

One pass, need by need, with three possible verdicts.

Already possible and we did not know. Possible after configuration — a field, a workflow, a permission, a template. Genuinely not achievable, because the tool's data model does not support it.

Five of ours landed in the first two categories. Status, versioning, execution separation, execution assignment, suite management, peer-review reporting — the friction was real, and it was ours.

One survived: there is no JQL over test cases. Test cases are not queryable the way issues are, which means every cross-cutting question — show me every case touching this area, in this status, unreviewed — has to be answered by export and manual work rather than a saved query.

That is not a preference. That is a property of the data model, and no amount of configuration reaches it.

What the Honest Business Case Looks Like

It is one gap, described precisely, with the cost of living with it attached.

That is a much smaller document than the one we were going to write, and it is far more likely to succeed — because it is falsifiable, it does not require anyone to accept that the last tool decision was wrong, and it can be answered with something other than a migration. Maybe the answer is a migration. Maybe it is a reporting workaround, or accepting the export overhead, or a plugin. A one-gap case lets the room choose. A wholesale swap forces them to either agree with all of it or reject all of it.

And the five needs that turned out to be configuration do not disappear from the exercise. They become a backlog — small, specific, assignable, each one a day of admin work rather than a quarter of migration. That backlog is the actual return on the desk-check, and you get it whether or not anyone ever buys anything.

Why Nobody Runs the Desk-Check

Because it requires admitting, in writing, that you might not know your own tool.

There is a real asymmetry here. The pain is felt daily by the people executing tests. The configuration knowledge sits with whoever set the tool up, often years ago, often no longer on the project. Nobody currently owns the question "could this be configured", so nobody asks it, and the frustration compounds until it is loud enough to become a procurement conversation.

Reading documentation for an afternoon is also, plainly, less satisfying than choosing a new tool. One feels like progress and the other feels like homework.

The rule is short enough to remember when the frustration next peaks:

Before you write the business case for a new tool, write the list of capabilities you actually need and check it against what you already own. The gap that survives configuration is the only thing worth migrating for.

Of the last three things you complained your test tool could not do — how many did you check?