What Is Test Evidence? Why 'Passing' Isn't the Same as 'Proven'

"This feature was tested" and "we have evidence this feature was tested" sound almost the same — but to an auditor, a client, or a new teammate picking up a module six months later, they're completely different things. The first is a sentence. The second requires a record that survives, and holds up under scrutiny. This article defines what test evidence actually is, what it's made of, and why the word "passing" was never evidence on its own.

What test evidence actually is

Test evidence is the record that answers a full chain of questions — which behavior of which feature was verified, by whom, when, against what standard, with what result — not a single pass/fail flag, but a connected set of information that holds up when someone checks it later.

"Passing" isn't the same as "proven"

A test case showing "pass" could mean any of these:

• The feature genuinely works correctly, and the case verifies it specifically.
• The case is actually verifying an old behavior that no longer exists (see test case drift).
• The case is written so vaguely it would pass regardless of whether the feature works.
• The case was never mapped to a real requirement, so what it verifies doesn't matter.

Looking at the "pass" result alone, there's no way to tell which of these is true. Real evidence needs enough context for someone reviewing it later to figure that out themselves.

What a piece of test evidence actually needs to contain

Component	Question it answers	What breaks without it
Linked requirement	Which specific requirement does this case correspond to?	No way to prove "what needed testing actually got tested"
A specific expected result	What counts as correct, what counts as wrong?	"Passing" might just mean the case was written vaguely
Execution record (who, when)	Who ran this verification, and when?	No way to answer "is this result still valid right now"
A preserved decision (if a human reviewed it)	If the case was reviewed, what was the reasoning?	The next person can't trust the decision and has to redo the work

Who actually asks to see test evidence

• Clients or auditors: "what did you test" isn't a casual question — it's often part of a contract or regulatory requirement.
• A new engineer or QA hire: picking up a module, evidence is the only way to reconstruct "how thoroughly was this actually tested" without needing to ask the previous owner.
• Whoever's investigating an incident after the fact: after something breaks in production, the first question is always "was this scenario tested, and what were the results."

What all three have in common: the person asking wasn't there, and can't re-run the original verification — they can only rely on what was actually recorded.

Why evidence has to be tied to a requirement

An execution record with no requirement link can only answer "this case ran" — not "this feature was verified." That's exactly the problem requirement traceability is meant to solve — traceability isn't for making reports look nice, it's what lets every execution record answer "what was this actually verifying."

How OpenTestX treats test evidence

OpenTestX treats reviewable evidence as a first-class concern — not just for traditional test cases, but for the behavior evals Eval Studio runs against AI features too, preserving a chain of evidence that keeps the requirement link, execution record, and reasoning behind every verification. When a requirement change triggers a case review, the reviewer's decision and rationale get preserved too — not just what the case looks like now.

FAQ

Does an automated test's execution log count as complete test evidence?

Depends on whether the record links back to a requirement and carries a timestamp. A plain "CI is green" screenshot, missing that context, doesn't count as complete evidence either.

Can a manually recorded test count as test evidence?

Yes, as long as it has the four components above. What makes something test evidence isn't whether it's automated — it's whether the record is complete enough to be independently verified.

Do small teams really need this level of formal test evidence?

It depends on the risk the feature touches and any external accountability requirements, not team size. Even a two- or three-person team should invest in evidence completeness if the product faces audits or important clients.

---

"It was tested" is a sentence. Test evidence is a record that can be re-examined and independently verified. Knowing the difference between the two is where judging whether a testing process is actually trustworthy begins.