O que é Action-value function, Bellman equation?

The action-value function of a policy is $Q^\pi(x, u) = \E_\pi[G_t\mid x_t = x, u_t = u]$, the expected cumulative reward of taking $u$ in $x$ and following $\pi$ afterwards; the value function of Book 4 is $V^\pi(x) = \sum_u\pi(u\mid x)Q^\pi(x, u)$. The Bellman equation is the recursion $Q^\pi(x, u) = \E[g_{t+1} + \gamma V^\pi(x_{t+1})\mid x, u]$; for the optimal policy, $Q^*(x, u) = \E[g_{t+1} + \gamma\max_{u'}Q^*(x_{t+1}, u')\mid x, u]$.