---
title: "Deep Learning"
url: "/learn/deep-learning"
---

Deep learning is often introduced as a break from what came before. It is not.
A neuron is a model from the previous category with its statistical
interpretation removed, and a network is a stack of them with a training
algorithm good enough to make the stack worth having.

Starting from that continuity makes the rest of the field much less mysterious.

## Subcategories

- [Foundations](/learn/deep-learning/foundations.md): Four posts, in dependency order: a neuron as a logistic regression with the statistics stripped out, backpropagation as the chain rule with enough bookkeeping to make training affordable, the non-linearities that make depth mean anything, and the short history of how one neuron became a network worth training.
- [Architectures](/learn/deep-learning/architectures.md): Four posts on the architectures that generalise the single neuron: convolution as weight sharing for spatial data, recurrence as weight sharing for sequence, and attention as the mechanism that replaced recurrence — building up to the transformer, block by block.
- [Training Dynamics](/learn/deep-learning/training-dynamics.md): Four posts on everything that sits between an architecture and a trained network: optimisers beyond plain gradient descent, batch normalisation, regularisation against memorising the training set, and the initial weights that decide whether training starts moving at all.
- [Generative Models](/learn/deep-learning/generative-models.md): Three posts on models trained to produce data rather than label it: autoencoders as compression used as a learning signal, generative adversarial networks as two networks trained against each other, and diffusion models as generation reframed as iterative denoising.