"Reviewed" Is Not a Verdict

A tick in a column marked Reviewed.

Somebody looked at the test case. That is everything that tick tells you. Not whether the case is executable, not whether a stranger could run it, not whether the reviewer agreed with the approach or simply ran out of afternoon.

Why Peer Review Dies

It dies for a boring reason: there is no shared language for the outcome.

A review with a binary flag gives the reviewer two options. Tick it, or start a conversation. Ticking is free. Starting a conversation means composing a message that says, in some tone you now have to invent, that a colleague's work is not good enough — with no established vocabulary to lean on and no agreed threshold to point at.

So the tick wins. Not out of laziness. Out of the absence of a middle setting.

And once the tick wins consistently, review becomes a scheduling ritual. Cases get flagged as reviewed, the flag means nothing, and the first genuine reader of the test case is whoever has to execute it under time pressure in the next sprint.

The Two Questions a Verdict Has to Answer

A verdict is only useful if it changes what happens next. Which means it has to answer two questions, not one.

Did the test case survive a dry run? And who holds the pen now?

The dry run is the anchor, and it is worth being precise about what it is. The reviewer executes the case exactly as written, against the system, without filling in anything from their own knowledge. Every gap they have to close from memory is a defect in the case. The dry run does not fail on the product — it fails on the test case, and that is what makes it an objective input rather than an opinion.

The second question decides the economics. If the reviewer fixes everything, the author learns nothing and review becomes a second author. If the reviewer fixes nothing, trivial typos travel back and forth for two days.

Four outcomes cover both questions.

The Four Tiers

• OMG! The case is unreadable. It does not get comments, it gets a rewrite — there is nothing stable enough underneath to give line-level feedback on, and pretending otherwise wastes both people's time.
• Try One More Time. Major issues: the dry run failed on the test case itself. Steps missing, preconditions absent, expected results that cannot be checked. Back to the author, with the failure point named.
• Close Enough. The dry run passed. What remains is typos, copy-paste leftovers, formatting — the kind of thing the reviewer fixes on the spot rather than sending back.
• Good Job! The dry run passed clean and there is nothing to change. This tier exists so that the good outcome has a name, and so the absence of feedback stops being ambiguous.

The line between the middle two is the one that does the work. It is not a judgement about quality. It is a mechanical question: did the case run as written? If yes, the reviewer finishes it. If no, the author does.

Why the Names Are Not a Joke

They look informal. That is the design.

A verdict has to be said out loud, in a stand-up, to a colleague, without a preamble apologising for it. "This one's a Try One More Time" is a sentence a person can actually say. "This test case does not meet the review criteria" is a sentence people write instead of saying, which is how feedback ends up in a document nobody reads.

Named tiers also survive a busy sprint. Criteria written in a wiki page do not — they get read once, during onboarding, and then approximated forever. A four-word vocabulary gets used because it is faster than the alternative, not because anyone is enforcing it.

And the humour is load-bearing in one specific place. Negative feedback between peers is socially expensive, and giving it a name that is already understood by both people removes most of that cost. OMG! is not an insult when it is a tier.

Running It So It Sticks

• Record the verdict under the test case, not in chat. One comment on the case in Jira, with the tier and the reason. That is the entire artefact, and it is what makes the review visible to anyone who reads the case later.
• Never fix a Try One More Time yourself. The author has to run into their own gap, because the gap is a habit, not an incident. Fixing it for them removes the only mechanism that changes the next case they write.
• State the failure point, not the fix. "The dry run stopped at step 4: no way to tell whether the expected result happened" teaches more than a corrected step 4, and it takes less time to write.
• Watch the tier distribution per author over a few sprints. A mentee moving from Try One More Time to Close Enough is a growth curve you can point at in a review conversation, and it is the only evidence of mentoring that survives the person leaving the project.

What This Really Replaces

A four-tier vocabulary sounds like process overhead. It is the opposite — it is what lets you stop doing something more expensive.

Without it, the only way a team catches bad test cases is at execution, under a deadline, when a case that cannot be run becomes an hour of someone reconstructing intent and then guessing. That hour is charged to the sprint, and it recurs on every future execution of the same case, and it is invisible in every report anybody produces.

With it, the same defect costs one dry run and one named verdict.

The point of the vocabulary is not to grade people. It is to make the review produce a decision — rewrite, retry, minor fix, or clean — because a review that does not produce a decision is a reading, and a reading is not a control.

Look at the last test case someone reviewed for you. Which of the four verdicts did they actually give — and could you tell from what they wrote?