Questions / Quant research, Backtest integrity
The best of many strategies
200 independent zero-skill variants each have a 2.275% chance of a t-statistic above 2. What is the probability at least one does?
Answer it in the app It's free. You get hints, a second try if you miss, and a follow-up question.
Show the answer and worked solution
Answer: 0.99 probability
- Each variant stays below 2 with probability 0.97725.
- All 200 stay below with 0.97725²⁰⁰ ≈ 0.010.
- At least one clears 2 with probability ≈ 0.99.
Interview shortcut. 1 − 0.97725²⁰⁰ ≈ 99%.
Takeaway. The best of many variants rewards noise.
In the app this has a follow-up where one thing about the setup changes. Try the follow-up.