O que é Markov chain, continuous-time Markov chain?

A Markov chain is a Markov process with a countable state space, observed at integer times, with transition matrix $P_{ij} = \P(X_{n+1} = j \mid X_n = i)$. A continuous-time Markov chain is a right-continuous Markov process $(X_t)_{t\ge0}$ with a countable state space whose transition probabilities $P_{ij}(t) = \P(X_{s+t} = j \mid X_s = i)$ do not depend on $s$.