Chapter 7: Symmetric Matrix. In the chapter mind map this icon labels Definiteness: $x^\top A x > 0$ (PD) / $\geq 0$ (PSD). The discussion below is excerpted and lightly edited from § Quadratic Forms and Definiteness in Mathematics for AI and Machine Learning.

Quadratic forms serve as the fundamental link between scalar-valued calculus and linear algebra. By encoding a second-degree polynomial into the compact form $\mathbf{x}^\top A \mathbf{x}$ (where $A$ is symmetric), we can leverage spectral theory to decode the function's high-dimensional geometry. The eigenvalues of $A$ are not merely algebraic numbers; they represent the principal curvatures of the landscape. They rigorously characterize the critical point as a convex 'valley' (positive definite, see below), a concave 'peak' (negative definite), or a 'saddle point' (indefinite, see below). In optimization, this structure underpins second-order methods, where the Hessian matrix locally approximates the objective function as a quadratic form to determine the optimal step direction. The connection between quadratic forms and the Hessian matrix becomes explicit through the second-order Taylor expansion (see the matrix chapter, equation \eqref{eq:taylor-multivariable}), where the quadratic term $\frac{1}{2}(\mathbf{x} - \mathbf{a})^\top H_f(\mathbf{x} - \mathbf{a})$ (with $H_f = \nabla^2 f(\mathbf{a})$ being the symmetric Hessian matrix) captures the local curvature of the function $f$ near point $\mathbf{a}$. This explains why positive definiteness of the Hessian guarantees a local minimum: the quadratic form is positive for all directions, indicating the function curves upward in all directions (a valley).