Chapter 11: Matrix Calculus. In the chapter mind map this icon labels Jacobian Matrix:$J_{ij} = \frac{\partial y_i}{\partial x_j}$. The discussion below is excerpted and lightly edited from § Jacobian Matrix in Mathematics for AI and Machine Learning.

The gradient provides derivatives for scalar-valued functions $f : \mathbb{R}^N \to \mathbb{R}$, giving us a column vector of partial derivatives. However, many important functions in machine learning and optimization are vector-valued, mapping $\mathbb{R}^N \to \mathbb{R}^M$ where $M > 1$. For example, neural network layers output vectors, and optimization algorithms often work with vector-valued objectives. The Jacobian matrix naturally extends the gradient concept to vector-valued functions by organizing the gradients of each output component as rows, creating an $M \times N$ matrix that captures how all $M$ outputs change with respect to all $N$ inputs simultaneously. This matrix representation is essential for the chain rule in vector calculus, backpropagation in neural networks, and understanding how transformations affect multidimensional spaces.