¿Qué es Newton’s method, quasi-Newton method, BFGS?

Newton’s method steps to the minimum of the local quadratic model, $x_{k+1} = x_k - \nabla^2f(x_k)^{-1}\nabla f(x_k)$. A quasi-Newton method replaces the Hessian by an approximation updated from gradient differences; the BFGS method (Broyden, Fletcher, Goldfarb and Shanno, 1970) updates the inverse approximation $H$ by $H \leftarrow (I - \rho sy^\top)H(I - \rho ys^\top) + \rho ss^\top$, with $s$ the step, $y$ the gradient change and $\rho = 1/y^\top s$; L-BFGS (Liu and Nocedal, 1989) keeps only the last few pairs.