Chapter 19: Stochastic Differential Equations. In the chapter mind map this icon labels SDEs as Limits of Optim Algos. The discussion below is excerpted and lightly edited from § SDEs as Limits of Optimization Algorithms in Mathematics for AI and Machine Learning.

The stochastic differential equations introduced provide the mathematical foundation for diffusion models. However, a remarkable fact is that the same probability distributions can be generated using deterministic ordinary differential equations.

Chapter the referenced section will show how to remove the noise entirely from SDEs while preserving the same marginal distributions. This leads to the concept of probability flow ODEs, which enable fast deterministic sampling from diffusion models. The key insight is that while individual sample paths differ between SDEs and ODEs, the distributional evolution is the same.

This connection between stochastic and deterministic processes reveals that diffusion models can be sampled using either:

• Stochastic methods (SDE solvers): Slower but more diverse samples
• Deterministic methods (ODE solvers): Faster but potentially less diverse samples

Finally, Chapter the referenced section will describe how probability distributions evolve over time through the Fokker-Planck equation, providing the highest-level mathematical description of diffusion processes. This PDE perspective unifies the path-level (SDE) and distribution-level (PDE) views of stochastic processes.