What we are building, and the plan

#The question

Some tasks cannot be answered by recalling text. A multi-step puzzle, a proof, a bug that needs tracing: the model has to work something out. Reasoning models do that by generating intermediate steps before the answer, and they get better in three ways.

1
A hard problem
2
Spend more computethink longer, sample many attempts, vote
3
Search with a verifiergenerate candidates, keep the ones that check out
4
Train with rewardsreinforce attempts that reach the right answer
5
A model that reasons better

#What the course builds

Part	Question it answers	You run
Reasoning Foundations	What is reasoning, and how do we measure it honestly?	A verifier, pass@k, a toy task
Reasoning at Inference	How much does extra compute buy?	Chain of thought, majority voting, best-of-N, verifier loops
Reasoning Training	How can a model learn to reason from rewards?	A GRPO-style training run on a toy problem
Lab	What does it cost per extra point of accuracy?	A cost versus accuracy simulation

Everything is runnable Python with no dependencies. The experiments are toy problems on purpose: you can see every number, and the mechanisms (voting, search, policy gradients) are the same ones used at scale.

#The route

Part	Lessons	Time	Checkpoint
Start here	3	about 1.5 hours	You can explain a policy, a reward and a baseline, and run the bandit example
Foundations	2	about 1 hour	You can write a verifier and compute pass@k
Inference	2	about 1.5 hours	You can show when majority voting helps and when it fails
Training	3	about 3 hours	The toy run's success rate climbs from about 0.2 towards 0.99 and you can say why

#What you will be able to do

Define reasoning operationally and measure it with a verifier.
Compare chain of thought, self-consistency, best-of-N and verifier loops on cost and accuracy.
Explain policy gradients, baselines and group-relative advantages, and run a toy version.
Read a reasoning-training recipe and spot the failure modes (reward hacking, collapse, wasted compute).

#Prerequisites, honestly

Transformer basics and sampling. Build an LLM from Scratch covers these. If you can say what a logit and a temperature are, you are fine.
Python. You will read short functions and run scripts.
Reinforcement learning: not needed. The next lesson is a primer.
Maths: averages, probabilities and a little calculus intuition. The maths tune-up covers what is used.

#Exercise

You have a task with a checkable answer (a maths problem with a known result). Name one way to use each of the three levers on it.

Answer: more compute: sample several attempts and take the majority answer. Search with a verifier: generate candidates and keep the ones whose final answer checks out. Rewards: use "answer is correct" as the reward and reinforce the attempts that earned it.

Next: reinforcement learning in one sitting.