A Markov chain is a memoryless stochastic process on a discrete state space in which the future state depends only on the current state, not on the history of how it was reached (the Markov/conditional-independence property). Key structural properties include irreducibility (every state reachable from every other), recurrence versus transience of individual states, and periodicity; for irreducible finite chains a stationary distribution — a probability vector S satisfying S·Q = S for transition matrix Q — always exists, is unique, equals the reciprocal of each state's expected return time, and (absent periodicity) is the limiting distribution of the chain regardless of initial conditions. Reversibility (detailed balance), the condition Sᵢqᵢⱼ = Sⱼqⱼᵢ, is a sufficient condition for S to be stationary and provides a direct, non-matrix-algebra method for finding stationary distributions in special cases such as random walks on undirected graphs, situating this material within the theory of stochastic processes as an extension of conditional probability to sequential, state-dependent randomness.
A Markov chain is a memoryless stochastic process on a discrete state space in which the future state depends only on the current state, not on the history of how it was reached (the Markov/condition…