The plan: roadmap, milestones and how to work through this course

A course like this is easiest when you can see the whole route. Here it is.

#The route

1
Start herethe picture, the plan, setup and prerequisites
2
Text to Tensorstokenizer, batches, embeddings
3
Attention and the GPTattention, the block, parameter counts
4
Training and Generationautograd, Adam, the tiny GPT lab, sampling, KV cache
5
Modern Upgrades and ScalingRoPE, RMSNorm, SwiGLU, GQA, LoRA, PyTorch mapping

#Part by part

Part	You build	Time	Checkpoint
0. Start here (4 lessons)	Nothing yet: the mental model, the plan, a working Python setup, and a quick tune-up of the maths you need	about 1 hour	`check.py` and the maths script run, and you can explain the pipeline in your own words
1. Text to Tensors (2 lessons)	A byte-pair-encoding tokenizer, then batches and embeddings	about 1.5 hours	Your tokenizer round-trips any string exactly, and you can say why `naïve café 🙂` costs more tokens than plain English
2. Attention and the GPT (2 lessons)	Self-attention with a causal mask, multi-head attention, the full block	about 1.5 hours	You can compute the exact parameter count of a GPT from its settings
3. Training and Generation (3 lessons)	Autograd, Adam, a 4,128-parameter GPT you train yourself, samplers and a KV cache	about 3 hours (the lab runs a couple of minutes)	Your loss falls from about 3.3 towards 2.2 and the model invents name-like strings
4. Modern Upgrades and Scaling (3 lessons)	RoPE, RMSNorm, SwiGLU, grouped-query attention, LoRA, and the mapping from your code to PyTorch	about 3 hours	You can read a Llama-style config and explain each field

Times are estimates for a reader who runs the code and does the exercises. Skimming is faster; learning it properly is slower.

#What you can skip

Comfortable with Python and the maths? Do the setup lesson, skim the prerequisites lesson and jump to Text to Tensors.
Already know tokenisation? Skim lesson 1 of Text to Tensors, do the exercise, and carry on.
Only want modern architecture? Parts 2 and 4 stand best on parts 1 and 3. Plan to read the first parts at least once quickly.

#A study routine that works

For every lesson:

Read the intro and the summary table so you know what the lesson is for.
Run each listing. Type it or paste it, and run it. Seeing the output beats imagining it.
Change one thing and predict the result. Make the vocabulary bigger, change the learning rate, remove the causal mask. Write down what you expect, then run it. A wrong prediction is the most valuable result: it points at the part of your model that needs fixing.
Answer the quiz without scrolling up. The explanations after each answer are part of the lesson.
Do the exercise on paper first, then check the answer.
Keep a notes file of numbers worth remembering: the vocabulary size, the parameter formula, the starting loss.

#Mistakes that waste time

Reading without running. The listings are short and quick to run.
Changing many things at once. You will not know which one mattered.
Skipping the quiz. It targets the usual misunderstandings.
Chasing scale. The tiny model is the point; do not try to train a bigger one in pure Python.
Getting stuck on the maths. If a derivation is hard, move on and return; the code is the ground truth.

#Where this leads

Finishing this course prepares you for the rest of the site: Build a Reasoning Model builds on your understanding of sampling and training, Fine-tuning and Customisation uses LoRA in anger, and the inference courses explain how the same model is served quickly and cheaply.

Next: set up your Python.