A/B Test Autopsy·interactive demo
Pathology · 05

Sample-ratio mismatch

A 50/50 split that arrives as 48/52 with 20k users is not noise. Any lift estimated from such a test is suspect. A χ² check at p < 10⁻³ catches assignment bugs before they reach analysis.

All 20 tests in this set

TestControl NTreatment N SRM pFlaggedTruly broken