O que é Stochastic gradient descent?

Stochastic gradient descent minimises $\frac1n\sum_if_i(x)$ by steps along the gradient of one randomly chosen term (or a small batch), $x_{k+1} = x_k - t_k\nabla f_{i_k}(x_k)$.