Wat is Nonlinear least squares, Gauss–Newton, Levenberg–Marquardt?

Nonlinear least squares minimises $\frac12\lVert r(x)\rVert^2$ for a residual vector $r$. The Gauss–Newton method approximates the Hessian by $J^\top J$, $J$ the Jacobian of $r$, and solves $J^\top J\,\Delta x = -J^\top r$. The Levenberg–Marquardt algorithm (Levenberg, 1944; Marquardt, 1963) damps it, $(J^\top J + \lambda\operatorname{diag}(J^\top J))\Delta x = -J^\top r$, raising $\lambda$ after a failed step (toward scaled gradient descent) and lowering it after a success (toward Gauss–Newton).