Low-Rank Adaptation
From the book
Chapter 4: QR Decomposition and Numerical Rank. In the chapter mind map this icon labels Rank of Sum & Product Bounds. The discussion below is excerpted and lightly edited from § Theorem: Rank of Product in Mathematics for AI and Machine Learning. Related material also appears in Chapter 9 (Low-Rank Adaptation (LoRA): $W = W_0 + BA$).
For $A \in \mathbb{R}^{M \times d}$ and $B \in \mathbb{R}^{d \times N}$,
What this drawing shows
What you see. Shows a full update approximated by thin low-rank factors, matching the LoRA idea of efficient parameter adaptation.
In the mind map. Chapter 4 — Rank of Sum & Product Bounds. See From the book above for definitions, figures, and worked examples.
Also appears in Ch. 9 (Low-Rank Adaptation (LoRA)).
Where to read next
Read the full definitions, figures, and worked examples in Chapter 4: QR Decomposition and Numerical Rank — see the mind-map node Rank of Sum & Product Bounds.
This concept is also referenced in Chapter 9 (Low-Rank Adaptation (LoRA)).
