Semua buku

Profesional

Aplikasi Tentang Pelatih Masuk Mulai membaca

Quantitative Finance · Glosarium

Apa itu Target leakage, train–test contamination?

Dikenal juga sebagai: target leakage · train--test contamination

Definition 3.5 Machine Learning for Markets · Bab 3 — Validation

Target leakage is information in a feature, or in any step of the pipeline, that would not have been available at the decision time the prediction stands for, typically because it was derived from the target or recorded after it. Train–test contamination is information about the test data reaching the fit through the data themselves: overlapping labels or duplicated records on both sides of a split, or statistics computed on the whole sample.

leakleaky Roos2R^2_{\mathrm{oos}}honest versioncaught by
period-end join of the surprise1.27%−0.19%-0.19\% (filing date)truncation test
target encoding, same month8.97%−0.19%-0.19\% (none)canary, truncation test
screening on the whole sample0.13%−0.16%-0.16\% (training months)canary, truncation test
shuffled folds, overlapping labels3.47%−3.37%-3.37\% (purged folds)fold-overlap check
best of 30 seeds on the test−0.27%-0.27\%−0.20%-0.20\% (untouched months)holdout, nested search
Table 3.1. Five leaks in a world where nothing is predictable at decision time (200 stocks, 240 months). Every positive number is fake; the last row’s leak is a gain of 0.13 points over the median seed that does not recur. Screening uses ridge regression on the ten screened features, the others boosted trees. Data: ml_validation.leaks.
Baca dalam konteks →