This is an advanced probability course dealing with discrete and continuous time Markov chains. The course covers the fundamental theory, and provides many examples. Markov chains has countless applications in many fields raging from finance, operation research and optimization to biology, chemistry and physics.