Questions / Quant research, Machine learning judgment
A misleading accuracy score
Only 1% of 10,000 observations are positive. A classifier always predicts negative. What is its accuracy?
Answer it in the app It's free. You get hints, a second try if you miss, and a follow-up question.
Show the answer and worked solution
Answer: 99% (99/100)
- There are 100 positive and 9,900 negative observations.
- Predicting negative every time correctly classifies the 9,900 negatives and misses all 100 positives.
- Accuracy is 9,900/10,000 = 99%, even though positive-class recall is zero.
Interview shortcut. Accuracy of the all-negative rule is 1 − prevalence.
Takeaway. Compare headline accuracy with the majority-class baseline.
In the app this has a follow-up where one thing about the setup changes. Try the follow-up.