Method
How a verdict is decided
The engine executes methods, computes statistics, and owns verdicts. Nothing on this page is a heuristic dressed up as a method: every axis is a genuine re-analysis and every sentinel flag carries a re-derivable statistic.
1 · The multiverse grid
A fixed grid of real methods reruns the finding. It survives an axis only if it stays significant AND same-direction. The grid is deterministic and byte-identical on re-run. When an axis cannot run honestly — for example donor-level pseudobulk without donor structure — it skips with a caveat rather than fabricating a number.
- QC thresholdcell/gene filtering choices
- Normalizationlibrary-size vs alternative normalization
- Design-matrix batch model~condition + batch
- DE testalternative differential-expression tests
- Benjamini–Hochberg significancecritical axis
- Leiden re-clusteringresolutions 0.25 / 0.5 / 1.0
- Harmony integrationreal batch integration
- Donor-level DESeq2 pseudobulkcritical axis when donors exist; skips honestly otherwise
2 · Confound sentinels
Structural checks on the experimental design. Each flag is backed by a statistic anyone can re-derive from the data — never a model opinion.
Batch / condition confounding
Evidence: Cramér's V between condition and batch
Pseudoreplication
Evidence: donors-vs-cells structure at the tested effect
Covariate confounding
Evidence: type-aware association (Cramér's V / correlation ratio η)
Group imbalance
Evidence: group-size ratio
Double-dipping
Evidence: grouping-source vs tested signal
Doublet-driven signal
Evidence: doublet score vs experimental HTO labels (real)
Knockdown failed
Evidence: perturbed-gene expression in the perturbed arm
Trivial effect size
Evidence: standardized effect (Cohen's d)
3 · Scoring & overrides
Robustness ≥ 0.9 across the grid, no critical-axis break, no fatal flag. The finding survives.
Robustness 0.6–0.9. The finding is fragile — it depends on defensible choices. Reported as at-risk, not fatal.
Robustness < 0.6, or a critical-axis break, or a severe structural confound, or a finding-fatal flag. Does not survive.
Critical-axis overrides
- A break on Benjamini–Hochberg significance, or on the donor-level pseudobulk axis when it ran, caps the verdict below green. A pseudobulk break is a pseudoreplication false positive — it cannot be outvoted by the wider grid.
- A severe structural confound (association statistic ≥ 0.9) or pseudoreplication forces red.
- A finding-fatal flag — trivial effect, doublet-driven, failed knockdown, double-dipping — takes the finding off green.
4 · Determinism & signing
All randomness is seeded, so the certificate is byte-identical on re-run. The signed certificate reconstructs and re-checks its own signature on every read — it never trusts a signature it did not itself recompute. A reproducibility bundle pins the data hash, the spec, and the dependency set for one-command reproduction.
scVI is deliberately excluded: it trains stochastically and would break byte-identical reproducibility.
See the method graded
The validation atlas shows exactly which cases survive, which fail, and on which axis — checked against planted ground truth.