Mathematics for AI and Machine Learning

Foundations for modern AI and machine learning

Gram-Schmidt

Chapter 3 Linear algebra

Chapter 3: Subspaces and Orthogonality — Null Space Basis via Gram-Schmidt (also appears in Ch. 4)

High-resolution PNG
Gram-Schmidt — high-resolution mind-map icon

From the book

Chapter 3: Subspaces and Orthogonality. In the chapter mind map this icon labels Null Space Basis via Gram-Schmidt. The discussion below is excerpted and lightly edited from § Algorithm: Null Space of in Mathematics for AI and Machine Learning. Related material also appears in Chapter 4 (Gram-Schmidt QR).

``{.algorithm caption="Null Space of $A^\top$" label="alg:null-space-at"} \Require Matrix $A \in \mathbb{R}^{M \times N}$ \Ensure An orthonormal basis of $N(A^\top)$ \State Perform QR decomposition: $A = QR$ \State $j \gets \mathrm{rank}(A)$ \For{$m = 0, 1, \ldots, M-1$} \State Let $\mathbf e_m \in \mathbb{R}^{M}$ be the $m$-th standard basis vector \State Project $\mathbf e_m$ onto existing orthonormal vectors: \State $r_{im} \gets \mathbf e_m^\top \mathbf q_i$ for $i = 0, \ldots, j-1$ \State Compute the rejection: $\mathrm{rej}(\mathbf e_m) \gets \mathbf e_m - \sum_{i=0}^{j-1} \mathbf q_i r_{im}$ \If{$\mathrm{rej}(\mathbf e_m) \neq 0$} \State $\mathbf q_j \gets \frac{\mathrm{rej}(\mathbf e_m)}{||\mathrm{rej}(\mathbf e_m)||_2}$ \State $j \gets j + 1$ \EndIf \If{$j = M$} \State \textbf{break} \EndIf \EndFor ``

What this drawing shows

What you see. Represents turning a list of vectors into an orthonormal basis by subtracting projections.

In the mind map. Chapter 3 — Null Space Basis via Gram-Schmidt. See From the book above for definitions, figures, and worked examples.

Also appears in Ch. 4 (Gram-Schmidt QR).

Where to read next

Open Chapter 3 companion →

Read the full definitions, figures, and worked examples in Chapter 3: Subspaces and Orthogonality — see the mind-map node Null Space Basis via Gram-Schmidt.

This concept is also referenced in Chapter 4 (Gram-Schmidt QR).