---
title: "Generative Models"
url: "/learn/deep-learning/generative-models"
category: "deep-learning"
---

Everything earlier in this category is trained to map an input to an output.
This subcategory is about the models trained to produce an output with nothing
supplied as input — and the three approaches here answer that with genuinely
different mechanisms rather than variations on one idea.

## Posts

- Autoencoders: Compression as a Learning Signal (unpublished): An autoencoder has no labels, no external objective beyond reconstructing what it was given — and the bottleneck it is forced through is what makes that reconstruction task a genuine learning signal. This post derives the architecture, the compression it performs, and how it generalises into the variational autoencoder.
- Diffusion Models, From First Principles (unpublished): Diffusion models start from a deliberately simple forward process — gradually adding noise until data becomes indistinguishable from static — and train a network to reverse it one small step at a time. This post derives the forward process, the reverse process it implies, and why generation by many small steps proved more stable than in one large one.
- Generative Adversarial Networks: Two Networks Playing a Game (unpublished): A generative adversarial network pits a generator, trying to produce convincing fake data, against a discriminator, trying to catch it, and trains both simultaneously as a minimax game. This post derives that game formally, the equilibrium it aims for, and the instability that made GANs notoriously difficult to train.