Todos os livros

Profissional

Apps Sobre Coach Entrar Começar a ler

Quantitative Finance · Glossário

O que é Target leakage, train–test contamination?

Também chamado de: target leakage · train--test contamination

Definition 3.5 Machine Learning for Markets · Capítulo 3 — Validation

Target leakage is information in a feature, or in any step of the pipeline, that would not have been available at the decision time the prediction stands for, typically because it was derived from the target or recorded after it. Train–test contamination is information about the test data reaching the fit through the data themselves: overlapping labels or duplicated records on both sides of a split, or statistics computed on the whole sample.

leakleaky Roos2R^2_{\mathrm{oos}}honest versioncaught by
period-end join of the surprise1.27%−0.19%-0.19\% (filing date)truncation test
target encoding, same month8.97%−0.19%-0.19\% (none)canary, truncation test
screening on the whole sample0.13%−0.16%-0.16\% (training months)canary, truncation test
shuffled folds, overlapping labels3.47%−3.37%-3.37\% (purged folds)fold-overlap check
best of 30 seeds on the test−0.27%-0.27\%−0.20%-0.20\% (untouched months)holdout, nested search
Table 3.1. Five leaks in a world where nothing is predictable at decision time (200 stocks, 240 months). Every positive number is fake; the last row’s leak is a gain of 0.13 points over the median seed that does not recur. Screening uses ridge regression on the ten screened features, the others boosted trees. Data: ml_validation.leaks.
Ler no capítulo →