Alle boeken

Professioneel

Apps Over Coach Inloggen Begin met lezen

Quantitative Finance · Begrippenlijst

Wat is Momentum method, Nesterov acceleration?

Ook bekend als: momentum method · Nesterov acceleration

Definition 24.3 Quantitative Methods · Hoofdstuk 24 — Numerical Optimisation in Practice

A momentum method adds a fraction of the previous step to the gradient step. Nesterov acceleration evaluates the gradient at an extrapolated point, yk=xk+k−1k+2(xk−xk−1)y_k = x_k + \frac{k-1}{k+2}(x_k - x_{k-1}), xk+1=yk−∇f(yk)/Lx_{k+1} = y_k - \nabla f(y_k)/L, which improves the convex rate to O(1/k2)O(1/k^2) and the strongly convex rate to about 1−1/κ1 - 1/\sqrt\kappa per iteration.

Gradient descent and Nesterov’s accelerated method on a quadratic with condition number = 100, step 1/L, from (1, 1). Gradient descent follows its linear rate exactly; the accelerated method is not monotone but is far faster. Data: the chapter’s tutorial.
Figure 24.1. Gradient descent and Nesterov’s accelerated method on a quadratic with condition number κ=100\kappa = 100, step 1/L1/L, from (1,1)(1, 1). Gradient descent follows its linear rate exactly; the accelerated method is not monotone but is far faster. Data: the chapter’s tutorial.
Lees in het hoofdstuk →