---
title: "Supervised Learning"
url: "/learn/machine-learning/supervised-learning"
category: "machine-learning"
---

Linear regression and logistic regression between them introduce the feature
vector, the weight vector, the loss, the gradient and the regulariser — the
full vocabulary that every model in this subcategory reuses.

Decision trees and the forests built from them approach the same problem from
a completely different direction: no gradient, no closed form, just recursive
splitting — and support vector machines return to an optimisation view, from
the geometry of the margin rather than the likelihood.

## Posts

- Decision Trees: How a Model Learns to Ask Good Questions (unpublished): Unlike every model derived so far in this corpus, a decision tree isn't fit by an optimiser — it's grown by recursively splitting on whichever question reduces impurity the most. This post derives the impurity measures behind that choice and where a greedy tree's structure comes from.
- Support Vector Machines: Maximizing the Margin, Geometrically (unpublished): Where logistic regression asks for a probability, a support vector machine asks a purely geometric question — what is the widest possible margin between two classes, and which points define it? This post derives the margin-maximisation objective, support vectors, and a first look at the kernel trick.
- From Trees to Forests: Bagging, Boosting, and Why Ensembles Win Competitions (unpublished): Random forests and gradient-boosted trees dominate classical machine learning competitions, and both are built from the same weak base learner combined two structurally different ways. This post derives bagging as variance reduction through averaging, and boosting as sequential error correction.