Five deterministic labs that isolate the statistical-reasoning traps every methods course teaches separately: regression-to-the-mean, survivorship bias, hot-hand run-length, base-rate/Bayes (the prosecutor's fallacy), and a p-hacking spec-curve multiverse. Every lab runs on your inputs plus static, cited built-ins, nothing live, nothing that needs refreshing.
Abraham Wald (Statistical Research Group, 1943) was asked where to add armor based on hit locations counted on bombers that RETURNED from missions. The military's instinct was to reinforce the most-hit sections. Wald reasoned the opposite: those sections show heavy damage precisely because planes hit there still made it home. The sections that matter are the ones with few or no hits on returning planes, because planes hit there likely never returned to be counted.
| Section | % of Returning Planes Hit Here | Wald's Armor Priority |
|---|
Generates a synthetic two-group dataset (n=20/group) from a seeded pseudo-random generator with a TRUE null effect: there is no real difference between the groups. It then runs all 16 "researcher degree of freedom" analysis paths (exclude/keep an outlier × control/ignore a covariate × full sample/subgroup × one-/two-tailed) and shows how many come back "significant" at p<.05 by chance alone. Same seed always produces the same dataset and the same 16 results, nothing here reads the clock or an unseeded random source.
| Exclude Outlier | Control Covariate | Subgroup Only | One-Tailed | p-value | Result |
|---|