The library
Long-form writing on the engineering discipline underneath data, machine learning and AI — derived from first principles, not summarised from elsewhere.
The library is organised two ways, and both are worth knowing about.
By taxonomy. Every post sits in exactly one category and one subcategory, which is where its URL comes from. If you know roughly what you are looking for, this is the faster route.
By series. A series is an ordered arc that cuts across the taxonomy — the maths, then the models, then the networks — and reads front to back. If you are learning something rather than looking something up, start with a series.
Posts that are written but not yet released appear here as cards without links. That is deliberate: the plan is public before the prose is finished.
Most RAG debugging starts at the prompt and the model, when the failure usually happened earlier — at chunking. This post runs six chunking strategies against the same 400-page corpus and scores each one on whether the right passage actually made it into the context window, with the eval harness and the golden set published alongside the results.
Writing the prompt before the eval is the same mistake as writing code before the test — it feels faster and costs more later. This post makes the case for treating the eval as the spec an AI feature is built against, and shows what teams actually pay when they add it after the feature has already shipped.
Gradient descent gets used long before it gets derived, and that gap is where the intuition for tuning it never forms. This post derives it by hand — the update rule, why the learning rate behaves the way it does, and where the method breaks — so it stops being a black box borrowed from a library.
The mathematics you actually need to read a paper, derive a model by hand, and recognise when a result is wrong.
Models that learn from data, derived from first principles: what each one assumes, what it optimises, and where it breaks.
Networks of simple units, and the algorithm that makes training them possible at all.
Retrieval, evaluation, and the rest of the engineering discipline that sits around a model once it has to run in production.
The full arc in eight parts: the mathematics, the two models that introduce the vocabulary, and the network that generalises them.
1 of 1 publishedThree posts that derive the same loss functions from two independent directions — likelihood and information — and show why they agree.
0 of 3 publishedFour posts on why combining weak models beats building one strong one, from the bias-variance tradeoff through forests to honest evaluation.
0 of 4 publishedFive posts on what turns an architecture into a network that trains — activations, initialisation, normalisation, regularisation, and the optimiser.
0 of 5 publishedThree posts tracing why recurrence was the obvious architecture for sequences, why it failed at scale, and what replaced it.
0 of 3 published