The Test Case That Passed and Still Broke Production

The Silence After the Release

There is a special kind of silence that falls over a project team on a Monday morning.

The release went out on Friday.
The dashboard shows 100% test pass rate.
The stakeholders signed off.

And yet, Production is currently exploding. Users are stuck, data is corrupt, and Support is flooding the Slack channel.

And everyone turns to look at QA.

"But the test cases passed. How did we miss this?"

We didn't miss the test case. We missed the reality.

The Green Checkbox Fallacy

In most ALM tools (Jira, Azure DevOps), a test case is binary:

• ✅ Passed
• ❌ Failed

Green feels safe. Green feels final. But in Salesforce, Green is often lying.

A passing test case doesn't mean the system is bulletproof. It only means that the specific sequence of clicks you performed did not result in an immediate error message in that specific environment.

Dissecting the Failure: TC-17

Let’s look at the "perfect" test case that failed to protect us.

TC-17: Verify that Sales Rep can create an Opportunity
Steps:Log in as Sales Rep.Click New Opportunity.Fill in required fields (Name, Stage, Close Date).Click Save.
Expected Result: Record saves successfully.

Actual Result: Pass ✅

QA did their job. The functionality works.

Except… what did this test actually verify?
It verified that a fresh user could create a fresh record in a vacuum.

What It Didn’t Test (But Production Did)

Production users don't live in a vacuum. They live in a messy, historical, interconnected ecosystem.

Production Reality:

• The Sales Rep was editing an Account owned by another user (Sharing Rule collision).
• The Account had data migrated from 2019 (Legacy Data issues).
• The "After-Save" Flow triggered an integration that timed out (Governor Limits).
• A validation rule fired because the "Stage" change conflicted with an old field update (Order of Execution).

Your Test Case:

• Used fresh data created 5 minutes ago.
• Used the "Happy Path" data inputs.
• Ignored record ownership.
• Ignored history.

Same feature. Different universe.

Salesforce Is State-Based, Not Step-Based

Classic software testing assumes: "If the steps work, the system works."

Salesforce laughs at that assumption. Salesforce is State-Based.

The outcome of clicking "Save" depends entirely on the state of the system before you clicked it:

• Record Type State: Which process is assigned?
• Ownership State: Who owns the parent record?
• Data State: What are the values of the fields you didn't touch?
• Automation State: What else is running in this transaction?

A script that lists "Click here, type there" ignores the context. And context is the only thing that matters.

The Passing Test Case Trap

The most dangerous test cases are the ones that always pass.

They look solid on reports. They increase your "Code Coverage" metrics. They make management feel good.

But they create confidence without coverage. If a test case hasn't failed in six months, it's probably not testing anything risky. It’s testing that the Salesforce "Save" button works. (Spoiler: Salesforce tests that for us).

The Question QA Should Ask Instead

We need to stop asking:

“Did the test pass?”

And start asking:

“What specific condition would make this fail in production?”

• Ownership? Test with a user who doesn't own the Account.
• Bulk? Test with 50 records via Data Loader.
• Old Data? Test with a record created before the new Validation Rule existed.
• Integration? Test what happens when the external system sends a 404 error.

That is where real testing starts.

A Better Test Case Mindset

We need to rewrite our definitions.

Old Way:

"Verify user can create record."

New Way:

"Verify system behavior when a restricted Sales Rep attempts to create an Opportunity on a strategic Account owned by a Manager."

Less steps. More thinking. More reality.

QA Takeaways

1. Pollute Your Data: Don't test on clean records. Test on "ugly" records with missing fields and strange histories.
2. Context Over Clicks: Document the State required for the test (User, Permissions, Data) more thoroughly than the Steps.
3. Review the Greens: Periodically audit your passing tests. If they are too simple, they are blinding you to the real risks.

Final Thought

A passed test case is not proof of quality. It’s just a log entry saying you tried.

In Salesforce, the system doesn’t break when you click buttons. It breaks when reality touches configuration.

And reality never follows your test steps.

Green does not mean safe. It just means you haven’t looked closely enough.