Questions / Quant research, Machine learning judgment
Select features inside the split
Before cross-validation, you use every row and label to select the 20 features most correlated with the target. What is the problem?
Answer it in the app It's free. You get hints, a second try if you miss, and a follow-up question.
Show the answer and worked solution
Answer: D. Held-out labels helped choose the feature set before scoring.
- Each held-out fold should represent observations unseen by every fitted part of the procedure.
- Here the global target-correlation ranking includes labels from every fold.
- Validation outcomes have therefore influenced which predictors are available to the estimator before evaluation.
Interview shortcut. Feature selection on all rows leaks labels into cross-validation.
Takeaway. Feature selection is part of fitting, not a harmless pre-evaluation filter.
Why the other choices fail
- None, if correlation may be weak overall. Using held-out labels leaks regardless of strength.
- Using correlation rather than regression keeps selection independent of labels. Correlation with the target explicitly uses labels. Its simplicity does not isolate held-out information.
- Feature selection only reduces dimension, so it cannot fit the target. Selecting features by their target correlations is a learned, label-dependent operation even when it reduces dimension.
In the app this has a follow-up where one thing about the setup changes. Try the follow-up.