Alle boeken

Professioneel

Apps Over Coach Inloggen Begin met lezen

Quantitative Finance · Begrippenlijst

Wat is Online learning, recursive least squares, forgetting factor?

Ook bekend als: online learning · recursive least squares · forgetting factor

Definition 12.2 Machine Learning for Markets · Hoofdstuk 12 — Online Learning and Drift

Online learning updates a model with each new observation, in constant time and memory, and is scored prequentially: each observation is predicted before it is learned from. Recursive least squares (RLS) updates the weighted least-squares solution β^t=arg⁡min⁡b∑s≤tλt−s(ys−xs⊤b)2\hat\beta_t = \arg\min_b\sum_{s\le t}\lambda^{t-s}(y_s - x_s^\top b)^2 by

kt=Pt−1xtλ+xt⊤Pt−1xt,β^t=β^t−1+kt(yt−xt⊤β^t−1),Pt=1λ(Pt−1−ktxt⊤Pt−1),k_t = \frac{P_{t-1}x_t}{\lambda + x_t^\top P_{t-1}x_t},\quad \hat\beta_t = \hat\beta_{t-1} + k_t(y_t - x_t^\top\hat\beta_{t-1}),\quad P_t = \frac{1}{\lambda}\bigl(P_{t-1} - k_tx_t^\top P_{t-1}\bigr),

where the forgetting factor λ∈(0,1]\lambda\in(0,1] discounts old observations; their weights sum to about 1/(1−λ)1/(1-\lambda), the estimator’s effective memory.

Prequential R-squared of recursive least squares (after 1 000 steps of warm-up) against its effective memory, for streams whose coefficients change every D steps on average. Data: ml_online.forgetting.
Figure 12.1. Prequential R-squared of recursive least squares (after 1 000 steps of warm-up) against its effective memory, for streams whose coefficients change every DD steps on average. Data: ml_online.forgetting.
Lees in het hoofdstuk →