---
title: "Calculus & Optimization"
url: "/learn/essential-maths/calculus-and-optimization"
category: "essential-maths"
---

A model is only as trainable as its loss function is differentiable. This
category takes that seriously: gradients as geometry, the chain rule as the
load-bearing idea of backpropagation, convexity as the property that decides
whether "converges" means anything, and constrained optimisation for the
problems that will not sit still as an unconstrained one.

Nothing here is specific to machine learning yet — that specificity starts in
the next two categories, once this vocabulary is in place.

## Posts

- The Chain Rule Is the Only Reason Deep Learning Works (unpublished): Depth would be useless without an efficient way to compute how the loss depends on an early layer's parameters. This post treats the chain rule not as a calculus exercise but as the load-bearing idea that makes an arbitrarily deep composition of functions differentiable in practice.
- Convexity, Local Minima, and Why Loss Landscapes Aren't as Scary as They Look (unpublished): Convexity is the property that makes "gradient descent converges to the best answer" a theorem rather than a hope. This post defines it precisely, shows what breaks in non-convex landscapes, and surveys the results explaining why non-convex training still tends to work in practice.
- Lagrange Multipliers and Constrained Optimization, From Scratch (unpublished): Regularisation, margin maximisation and several classical models are all constrained optimisation problems in disguise. This post derives the method of Lagrange multipliers from geometric first principles, and shows how a constrained problem becomes an unconstrained one with an extra variable.