Markov Chain
From the book
Chapter 18: Langevin Dynamics and Sampling. In the chapter mind map this icon labels Markov Chain Monte Carlo (MCMC). The discussion below is excerpted and lightly edited from § Markov Chain Monte Carlo (MCMC) in Mathematics for AI and Machine Learning. Related material also appears in Chapter 14 (Markov Chains & Stochastic Matrices).
We now place Langevin dynamics within the broader framework of Markov Chain Monte Carlo (MCMC) methods. This provides a more general theoretical foundation and helps us understand how Langevin dynamics relates to other sampling techniques.
What this drawing shows
What you see. Two states $S_1, S_2$ with fixed transition arcs; a red token hops $S_1\!\to\!S_2$ then $S_2\!\to\!S_1$, illustrating memoryless Markov transitions.
In the mind map. Chapter 18 — Markov Chain Monte Carlo (MCMC). See From the book above for definitions, figures, and worked examples.
Also appears in Ch. 14 (Markov Chains & Stochastic Matrices).
Where to read next
Read the full definitions, figures, and worked examples in Chapter 18: Langevin Dynamics and Sampling — see the mind-map node Markov Chain Monte Carlo (MCMC).
This concept is also referenced in Chapter 14 (Markov Chains & Stochastic Matrices).
