Wat is Temporal-difference learning, Q-learning?

Temporal-difference learning updates an estimate of a value towards a target built from the next reward and the current estimate of the next state’s value, $V(x_t)\leftarrow V(x_t) + \eta_\alpha\delta_t$ with $\delta_t = g_{t+1} + \gamma V(x_{t+1}) - V(x_t)$, without waiting for the end of the episode. Q-learning applies it to action values with the target $g_{t+1} + \gamma\max_{u'}Q(x_{t+1}, u')$, which learns the optimal policy’s values while acting otherwise (Watkins and Dayan, 1992); SARSA uses the action actually taken next, and learns the values of the policy it follows.