Grade Every Claim Next to the Claim

In one day of analysis I reached six conclusions and later withdrew all six.

Five of them came from the same mistake: I reasoned about behaviour from what the screen was doing, instead of reading what the code says.

Every one of those five was internally consistent with all the evidence available when I made it. That is the part worth sitting with. They were not careless. They were plausible, and plausible is the problem.

Confident and Correct Look Identical

From the outside, a confident QA finding and a correct QA finding are the same object. Same tone, same specificity, same authority in a stand-up.

What separates them is not intelligence. It is provenance — whether the claim was read from source, observed in the org, or inferred from a screen.

And provenance decays fast. You know exactly where a claim came from at the moment you make it. Two hours later you remember the claim. By the next morning it is simply something you know, and nobody in the room can tell which of your statements came from Apex and which came from watching a page render.

That is how a UI guess gets laundered into a stated fact, and it happens to careful people, not careless ones.

The Fix: Grade the Claim Where the Claim Lives

The remedy is unglamorous. Next to every claim in an analysis, put its evidence grade.

• CODE — read in source. The strongest grade available, and still not proof that the code runs in the org you are testing.
• ORG — observed in metadata or a record query. The record is the fact.
• UI — observed on screen. This is an input to a hypothesis, never a conclusion.
• DERIVED — inferred from other claims. Inherits the weakest grade among its inputs.
• UNVERIFIED — believed, unchecked. Legitimate to write down, provided it is labelled.

Two rules make the system work.

Grade the claim next to the claim, not in your head. A grade you have to reconstruct later is a grade you will reconstruct generously.

Never promote a UI observation to a conclusion. What you saw on screen is real — it is just evidence about the rendering, and the rendering is not the record.

Every Theory Must Predict Something Untested

The second half of the discipline is what stops a well-graded analysis from being a well-graded fiction.

Every theory has to make one prediction you have not yet checked. Then you go and check that one.

A theory that only explains what you have already seen is unfalsifiable and therefore worthless as an oracle — and you can build a beautifully coherent one out of five UI observations. Forcing a prediction converts an explanation into a test. Either it survives contact with the record, or it dies cheaply, in your own notes, before it reaches anyone else's plan.

Most theories die there. That is the system working, not failing.

Keep the Claims You Withdrew

This is the part everyone skips, and it turns out to be the most valuable artefact of the whole exercise.

Do not delete a withdrawn conclusion. Keep it, with the grade it originally carried and the reason it fell over.

Those six withdrawn claims map exactly where the evidence in this system is misleading. Five of them clustered on one thing: the UI presents states the records do not hold. That is not six mistakes — it is one property of the system, discovered six times.

The next person to analyse this feature will make the same five inferences, because the screen invites them. A register of withdrawn claims is the only artefact that warns them, and it is worth more than the conclusions that survived.

What This Costs and What It Buys

The cost is a column. Perhaps ten seconds per claim, and the mild discomfort of writing UNVERIFIED next to something you are fairly sure about.

What it buys is that your findings become auditable — by you first, and by anyone who challenges them second.

There is also a quieter benefit. When a developer pushes back on a finding graded CODE, with the class and method named, the conversation is short and technical. When they push back on a finding that was really a UI inference dressed as a fact, you lose — and you should, and the next finding you raise starts from a worse position regardless of its quality.

Grading is not bureaucratic caution. It is how a tester keeps the right to be believed.

Of the last five things you said confidently about a system, how many could you trace to source right now?