Circular inference
Testing the groups you defined
The stated claim
“GENE_1999 differs between the two clusters — a robustness score of 95% across pipelines.”
benchmark case S2_double_dippingDeciding evidence
The groups were defined from the same expression now being tested. A high robustness score here is not evidence — it is the circularity reproducing itself.
Finding-fatal flag: double-dipping. The engine detects the grouping source is the tested signal and takes the finding off green regardless of the grid.
Double-dipping (circular inference) is when you use the data to define groups, then test those groups on the same data. The test is guaranteed to find a difference — you built the groups to have one. A high multiverse-survival rate does not fix this; it just shows the circularity is stable.
What the engine does
A suspiciously robust finding
The claim survives nearly every axis. On its face, that reads as a strong green.
VERITAS inspects the grouping source
The engine checks how the compared groups were defined and finds they were derived from the very expression being tested.
Robustness is reinterpreted
Because the comparison is circular, the survival rate is not evidence of a real effect — the grid is re-testing a foregone conclusion.
The fatal flag caps the verdict
Double-dipping is finding-fatal: it takes the finding off green no matter how the multiverse votes. Verdict: DOES NOT SURVIVE.
What the scientist does next
The scientist splits the data — define groups on one half, test on the other — before treating the difference as real.
Verify your own finding
Run a bundled sample of this exact failure mode in the console, or open the validation atlas to see how it is graded.