Entropy
From the book
Chapter 14: Information Theory. In the chapter mind map this icon labels Entropy: $H(X) = -\sum p_i \log p_i$. The discussion below is excerpted and lightly edited from § Definition: Entropy (Discrete) in Mathematics for AI and Machine Learning.
For a discrete random variable $X$ with probability mass function $p(x) = P(X = x)$, the entropy is defined as the expected information content:
where the sum is over all possible values of $X$, and we use the convention $0 \log 0 = 0$.
What this drawing shows
What you see. Represents uncertainty, spread, or information content in a probability distribution.
In the mind map. Chapter 14 — Entropy. See From the book above for definitions, figures, and worked examples.
Where to read next
Read the full definitions, figures, and worked examples in Chapter 14: Information Theory — see the mind-map node Entropy.