Todos los libros

Profesional

Apps Acerca de Coach Iniciar sesión Empezar a leer

Quantitative Finance · Glosario

¿Qué es Multiple testing, data snooping, garden of forking paths?

También llamado: multiple testing · data snooping · garden of forking paths

Definition 12.7 Quantitative Methods · Capítulo 12 — Testing and Multiple Testing

Multiple testing is running many tests and reporting some of them. Data snooping is choosing a model, a strategy or a parameter by searching over the same data that then test it, so that the reported statistic is the best of a search presented as a single test. The garden of forking paths is the implicit version: every choice the analyst would have made differently had the data been different (the sample period, the universe, the filter, the cost model) multiplies the tests actually run, even when only one analysis is ever performed.

Probability that the largest t-statistic reaches c when nothing works: one test, the two hundred momentum variants of the tutorial (Gaussian simulation with their estimated correlation), and two hundred independent tests. At the best variant’s 3.11 (vertical line): 0.0009, 0.030 and 0.17. Data: the chapter’s tutorial, seeded.
Figure 12.2. Probability that the largest tt-statistic reaches cc when nothing works: one test, the two hundred momentum variants of the tutorial (Gaussian simulation with their estimated correlation), and two hundred independent tests. At the best variant’s 3.11 (vertical line): 0.0009, 0.030 and 0.17. Data: the chapter’s tutorial, seeded.
The t-statistics of the two hundred trend rules on one simulated ten-year random walk, against their lookback, with three one-sided 5% thresholds: a single test (1.645), the max-statistic critical value for this correlated family (2.92) and Bonferroni’s (3.48). The best rule, at 42 days, has t = 3.11. Data: the chapter’s tutorial, seeded.
Figure 12.3. The tt-statistics of the two hundred trend rules on one simulated ten-year random walk, against their lookback, with three one-sided 5% thresholds: a single test (1.645), the max-statistic critical value for this correlated family (2.92) and Bonferroni’s (3.48). The best rule, at 42 days, has t=3.11t = 3.11. Data: the chapter’s tutorial, seeded.
Leer en el capítulo →