Fisher Divergence
Chapter 17: Score Function and Energy-Based Models — Fisher Divergence Score Matching
From the book
Chapter 17: Score Function and Energy-Based Models. In the chapter mind map this icon labels Fisher Divergence Score Matching. The discussion below is excerpted and lightly edited from § Fisher Divergence Score Matching in Mathematics for AI and Machine Learning.
The key insight is to use integration by parts to eliminate the unknown density from the objective. Instead of minimizing the distance between score functions directly, we minimize a related quantity that can be estimated from samples.
What this drawing shows
What you see. True score field $s$ (blue) fixed; red $s_\theta$ arrows shrink inward and the purple gap closes as the Fisher objective is minimized.
In the mind map. Chapter 17 — Fisher Divergence Score Matching. See From the book above for definitions, figures, and worked examples.
Where to read next
Read the full definitions, figures, and worked examples in Chapter 17: Score Function and Energy-Based Models — see the mind-map node Fisher Divergence Score Matching.
