Questions / Quant research, Data & feature research

Missing outcomes can depend on the outcome

Challenge, about 10 minutes. Question R291.

A vendor reports execution shortfall only for orders that complete within ten minutes. Slow, expensive executions are more likely to remain unreported. You need mean shortfall for all submitted orders. Does averaging reported rows identify that target without more assumptions?

  1. ANo; averaging works once more than half of orders are reported.
  2. BYes; weight each reported row by the overall completion fraction.
  3. CYes; divide by all submitted orders.
  4. DNo; reporting selects orders partly by their cost.

Answer it in the app It's free. You get hints, a second try if you miss, and a follow-up question.

Show the answer and worked solution

Answer: D. No; reporting selects orders partly by their cost.

  1. The observed mean estimates cost among promptly completed, reported orders.
  2. Slow expensive executions are disproportionately absent, so that group need not represent submissions.
  3. Recovering the all-order mean requires additional outcomes or justified assumptions about missingness.

Interview shortcut. Outcome-dependent reporting biases the mean.

Takeaway. Outcome-dependent reporting changes the population your average represents.

Why the other choices fail

  • No; averaging works once more than half of orders are reported. No reporting rate removes selection on cost. The missing orders are disproportionately the expensive ones.
  • Yes; weight each reported row by the overall completion fraction. A single completion fraction cannot correct selection depending on the unobserved execution costs.
  • Yes; divide by all submitted orders. This assigns zero cost to unreported orders and does not recover their missing outcomes.

In the app this has a follow-up where one thing about the setup changes. Try the follow-up.

More data & feature research questions