---
title: "Linear Algebra"
url: "/learn/essential-maths/linear-algebra"
category: "essential-maths"
---

Everything a model does to data is a linear map until the very last non-linearity,
so this is where the vocabulary for describing "what a layer can and cannot do"
comes from.

Read in order: a vector as a point in space, a matrix as that map written down,
the two directions that survive it unchanged, the decomposition that generalises
those directions to any matrix, and the calculus needed to differentiate all of it.

## Posts

- Eigenvalues and Eigenvectors: The Directions a Transformation Doesn't Bend (unpublished): Every matrix has a handful of special directions it only stretches or shrinks, never rotates — its eigenvectors — and the amount by which it does so — its eigenvalues. This post derives both from the definition, and shows why they describe what a matrix does independent of the basis it was written in.
- Matrix Calculus for Backpropagation: Deriving Gradients by Hand (unpublished): Backpropagation is matrix calculus applied at scale, and skipping the derivation is why it stays a black box. This post works through differentiating with respect to a matrix, the shapes that fall out of each rule, and the Jacobian bookkeeping a deep learning framework performs on your behalf.
- The Singular Value Decomposition, and Why It's the Workhorse of ML (unpublished): The singular value decomposition factors any matrix, square or not, into a rotation, a scaling and another rotation. This post derives what that factorisation means geometrically, why it always exists where eigendecomposition doesn't, and previews where it resurfaces — PCA, low-rank approximation, and recommender systems.