3. Markov chain: discret-time Markov chains, classification of states, ergodic, stationary distribution. Continuous-time Markov chains, birth and death process.

• 马尔可夫链：离散时间的马尔可夫链，状态的分类，遍历性，平稳分布。连续时间的马尔可夫链，生灭过程。