Questions / Quant research, Backtest integrity
Choose the universe before seeing survivors
You backtest from 2010 using only stocks still listed today. What bias can this introduce?
Answer it in the app It's free. You get hints, a second try if you miss, and a follow-up question.
Show the answer and worked solution
Answer: B. Universe membership is conditioned on surviving until today.
- A company’s presence in today’s listing is partly determined by events after 2010.
- Using that future outcome to choose the 2010 universe excludes firms that later disappear.
- The bias arises from selection, even with accurate individual prices; its direction depends on the strategy and outcomes.
Interview shortcut. Today’s listing conditions on the future: survivorship bias.
Takeaway. Audit how a dataset’s membership was chosen, not only its price fields.
Why the other choices fail
- Historical price accuracy eliminates selection bias in the universe. Accurate prices do not repair future-dependent selection of which firms enter the dataset.
- None, if delisted firms may be a small share. Survivors are selected on the outcome, whatever their share.
- Dropping delisted stocks removes only observations after their delisting. Using only today’s listings can also remove those firms’ valid pre-delisting observations from the historical universe.
In the app this has a follow-up where one thing about the setup changes. Try the follow-up.