Alle boeken

Professioneel

Apps Over Coach Inloggen Begin met lezen

Quantitative Finance · Begrippenlijst

Wat is Multiple testing, data snooping, garden of forking paths?

Ook bekend als: multiple testing · data snooping · garden of forking paths

Definition 12.7 Quantitative Methods · Hoofdstuk 12 — Testing and Multiple Testing

Multiple testing is running many tests and reporting some of them. Data snooping is choosing a model, a strategy or a parameter by searching over the same data that then test it, so that the reported statistic is the best of a search presented as a single test. The garden of forking paths is the implicit version: every choice the analyst would have made differently had the data been different (the sample period, the universe, the filter, the cost model) multiplies the tests actually run, even when only one analysis is ever performed.

Probability that the largest t-statistic reaches c when nothing works: one test, the two hundred momentum variants of the tutorial (Gaussian simulation with their estimated correlation), and two hundred independent tests. At the best variant’s 3.11 (vertical line): 0.0009, 0.030 and 0.17. Data: the chapter’s tutorial, seeded.
Figure 12.2. Probability that the largest tt-statistic reaches cc when nothing works: one test, the two hundred momentum variants of the tutorial (Gaussian simulation with their estimated correlation), and two hundred independent tests. At the best variant’s 3.11 (vertical line): 0.0009, 0.030 and 0.17. Data: the chapter’s tutorial, seeded.
The t-statistics of the two hundred trend rules on one simulated ten-year random walk, against their lookback, with three one-sided 5% thresholds: a single test (1.645), the max-statistic critical value for this correlated family (2.92) and Bonferroni’s (3.48). The best rule, at 42 days, has t = 3.11. Data: the chapter’s tutorial, seeded.
Figure 12.3. The tt-statistics of the two hundred trend rules on one simulated ten-year random walk, against their lookback, with three one-sided 5% thresholds: a single test (1.645), the max-statistic critical value for this correlated family (2.92) and Bonferroni’s (3.48). The best rule, at 42 days, has t=3.11t = 3.11. Data: the chapter’s tutorial, seeded.
Lees in het hoofdstuk →