Alle boeken

Professioneel

Apps Over Coach Inloggen Begin met lezen

Quantitative Finance · Begrippenlijst

Wat is Probability of backtest overfitting, combinatorially symmetric cross-validation?

Ook bekend als: probability of backtest overfitting · combinatorially symmetric cross-validation

Definition 20.2 Research Craft: Predictors, Backtests, Measurement, Portfolios · Hoofdstuk 20 — Overfitting

The probability of backtest overfitting (PBO) of a selection procedure is the probability that the strategy it selects as the best in sample performs below the median of the candidates out of sample. Combinatorially symmetric cross-validation (CSCV) estimates it by cutting the history into SS blocks and, for each choice of S/2S/2 blocks as the in-sample set, ranking the in-sample winner among all candidates on the other half; the PBO is the share of splits where its rank is below the median (Bailey, Borwein, López de Prado and Zhu).

CSCV on the thousand trend systems’ search years (16 blocks, 3 000 splits): the distribution of the logit of the in-sample winner’s out-of-sample rank. Mass to the left of zero is the probability of backtest overfitting. Data: rs_overfit on synthetic returns.
Figure 20.1. CSCV on the thousand trend systems’ search years (16 blocks, 3 000 splits): the distribution of the logit of the in-sample winner’s out-of-sample rank. Mass to the left of zero is the probability of backtest overfitting. Data: rs_overfit on synthetic returns.
In-sample (17.9 years) against out-of-sample (5 years) Sharpe ratios of the trend systems (every fourth of the thousand). On noise the clouds are unrelated (correlation -0.05); with the trend they correlate at 0.33, yet the top in-sample system is not near the top out of sample. Data: rs_overfit.systems.
Figure 20.2. In-sample (17.9 years) against out-of-sample (5 years) Sharpe ratios of the trend systems (every fourth of the thousand). On noise the clouds are unrelated (correlation −0.05-0.05); with the trend they correlate at 0.33, yet the top in-sample system is not near the top out of sample. Data: rs_overfit.systems.
Lees in het hoofdstuk →

Verwante begrippen