Subcategory
Autoencoders, GANs and diffusion models — three different answers to how a network learns to generate rather than only classify.
Everything earlier in this category is trained to map an input to an output. This subcategory is about the models trained to produce an output with nothing supplied as input — and the three approaches here answer that with genuinely different mechanisms rather than variations on one idea.
An autoencoder has no labels, no external objective beyond reconstructing what it was given — and the bottleneck it is forced through is what makes that reconstruction task a genuine learning signal. This post derives the architecture, the compression it performs, and how it generalises into the variational autoencoder.
Diffusion models start from a deliberately simple forward process — gradually adding noise until data becomes indistinguishable from static — and train a network to reverse it one small step at a time. This post derives the forward process, the reverse process it implies, and why generation by many small steps proved more stable than in one large one.
A generative adversarial network pits a generator, trying to produce convincing fake data, against a discriminator, trying to catch it, and trains both simultaneously as a minimax game. This post derives that game formally, the equilibrium it aims for, and the instability that made GANs notoriously difficult to train.