Training and Generation

Autograd, Adam, a runnable tiny GPT, sampling strategies and the KV cache.

1. 1▶ Up nextPro22 min+50 XPAutograd and the training loop: loss, backpropagation and AdamBuild a scalar automatic-differentiation engine, verify its gradients numerically, then use it to understand…
2. 🧪LabPro60 min+100 XPLab: train a tiny GPT in pure PythonRun a complete 4,128-parameter GPT, with tokenizer, attention, MLP, autograd, Adam and sampling, in one…
3. 3Pro16 min+50 XPGeneration: sampling strategies and the KV cacheTurn probabilities into text: greedy, temperature, top-k and top-p with tested code; why generation is slow without a…