---
title: "Machine Learning"
url: "/learn/machine-learning"
---

Most machine learning writing starts at the API and works outwards. This starts
at the objective function and works towards the API, because the interesting
part of a model is what it is trying to minimise and what it had to assume to
get there.

The models here are the classical ones, which is the point — they are small
enough to derive completely, and everything larger is a variation on them.

## Subcategories

- [Foundations](/learn/machine-learning/foundations.md): Five posts on the ideas every later model in this category quietly assumes: a formal definition of learning, gradient descent derived by hand, the bias-variance tradeoff, the discipline of splitting data into train, validation and test, and the diagnosis of overfitting and underfitting from real curves.
- [Supervised Learning](/learn/machine-learning/supervised-learning.md): Five posts from the two models that introduce the vocabulary of supervised learning — linear and logistic regression — through decision trees and the ensembles built from them, to the margin-maximising geometry of a support vector machine.
- [Unsupervised Learning](/learn/machine-learning/unsupervised-learning.md): Three posts on learning from unlabelled data: k-means and hierarchical clustering as two different answers to 'what groups are actually in this data', and principal component analysis as dimensionality reduction reframed as reconstruction.
- [Model Evaluation & Validation](/learn/machine-learning/model-evaluation.md): Three posts on the question that comes after a model is fit: whether it is any good, measured properly. Precision and recall for when accuracy hides the cost of being wrong, cross-validation for estimating performance without lying to yourself, and calibration for whether a model's confidence means anything.