Mutual Information
From the book
Chapter 14: Information Theory. In the chapter mind map this icon labels Mutual Information: $I(X;Y)$. The discussion below is excerpted and lightly edited from § Definition: Mutual Information in Mathematics for AI and Machine Learning.
For two random variables $X$ and $Y$ with joint distribution $p(x, y)$ and marginals $p(x)$ and $p(y)$, the mutual information is
For continuous random variables, replace sums with integrals.
What this drawing shows
What you see. Represents shared information between random variables.
In the mind map. Chapter 14 — Mutual Information. See From the book above for definitions, figures, and worked examples.
Where to read next
Read the full definitions, figures, and worked examples in Chapter 14: Information Theory — see the mind-map node Mutual Information.