Chapter 17: Score Function and Energy-Based Models. In the chapter mind map this icon labels Geometry: Attractors & Vector Flows. The discussion below is excerpted and lightly edited from § Motivation: From Density to Geometry in Mathematics for AI and Machine Learning.

The preceding chapters (14–16) developed a framework centered on probability distributions, Kullback-Leibler (KL) divergence, and variational objectives such as the evidence lower bound (ELBO). A fundamental assumption underlying these constructions is the availability of a probability density function $p(\mathbf{x})$ or, at minimum, its log-likelihood $\log p(\mathbf{x})$.

In practice, however, many modern generative models encounter significant limitations and challenges that invalidate this assumption. Specifically, the density $p(\mathbf{x})$ often proves intractable to compute directly, the normalization constant required for proper probability normalization remains unknown, and only local information about the underlying distribution is accessible through practical means.

Here, local information refers to properties of the distribution that can be evaluated at specific points (or in small neighborhoods around points) without requiring knowledge of the entire distribution's global structure. Unlike global properties (e.g., the normalization constant, which requires integrating over the entire support), local information describes how the distribution behaves at a particular location $\mathbf{x}$ in the input space.