Variational Family
Chapter 16: Variational Inference and Latent Variables — Variational Family
From the book
Chapter 16: Variational Inference and Latent Variables. In the chapter mind map this icon labels **Formulations: Variational Family $q(\mathbf{z})$. The discussion below is excerpted and lightly edited from § Variational Family** in Mathematics for AI and Machine Learning.
1. Mean Field: $q(\mathbf z) = \prod_{i=0}^{k-1} q_i(z_i)$ (factorized, independent components) 2. Gaussian: $q(\mathbf z) = \mathcal{N}(\mathbf z | \boldsymbol\mu, \boldsymbol\Sigma)$ (full covariance or diagonal) 3. Neural Networks: $q_{\boldsymbol\phi}(\mathbf z | \mathbf x) = \mathcal{N}(\mathbf z | \boldsymbol\mu_\phi(\mathbf x), \boldsymbol\Sigma_\phi(\mathbf x))$ where $\boldsymbol\mu_\phi, \boldsymbol\Sigma_\phi$ are neural networks (used in VAEs)
What this drawing shows
What you see. Candidate $q_\phi(z)$ curves fixed; highlight cycles across family members until one approximation becomes solid with filled support.
In the mind map. Chapter 16 — Formulations: Variational Family. See From the book above for definitions, figures, and worked examples.
Where to read next
Read the full definitions, figures, and worked examples in Chapter 16: Variational Inference and Latent Variables — see the mind-map node Formulations: Variational Family.