The plan and the map of the code

#The route

1
Start herepicture, plan, setup, a short C primer
2
Engine Coreforward pass, weight loading, KV cache and sampling
3
Engine Fastquantised matmul, threads and SIMD, batching and a server
4
Labbenchmark against llama.cpp

Part	Lessons	Time	Checkpoint
Start here	3	about 1 hour	The C benchmark compiles and runs; you can describe the engine's pieces
Engine Core	3	about 3 hours	Cached decoding gives the same logits as uncached; your model file round-trips
Engine Fast	4	about 4 hours	INT8 matmul error is small and you can explain the speed-up; you can serve the engine over HTTP

Times assume you run the code and do the exercises.

#The map of the code

1
server.pyan HTTP server that calls the engine for each request
2
batching.pya simulation of static versus continuous batching
3
quant.pyINT8 per-row and grouped 4-bit weights, and a quantised matvec
4
gguf_mini.pywrite and read a minimal GGUF-style file, memory-mapped
5
engine.pyConfig, weights, RMSNorm, RoPE, attention, KV cache, SwiGLU, generate

Reading from the bottom up gives the dependency order: the engine stands alone; the file format and quantisation work on its weights; batching and the server sit on top. A short C file, matvec_bench.c, measures memory-bound matrix-vector products.

#How a request flows through the files

server.py receives a prompt and hands the token ids to the engine.
engine.py calls forward_token once per token, reading the weights and the KVCache.
Each step returns logits; the sampler picks a token; the loop repeats.
If the weights came from a file, gguf_mini.py loaded them with memory-mapping.
If they are quantised, quant.py supplies the matrix multiply.

#Study routine for code-reading courses

Read the lesson's explanation first, then the code.
Run it on a tiny input and print intermediate shapes and values.
Change one thing and predict the result: width, number of heads, the quantisation group size.
Check the test: every part has an equivalence check (cached equals uncached, quantised close to float).
Write down the numbers that matter: memory per token, bytes per weight, tokens per second.

#What you can skip

Know transformers well? Skim the Engine Core forward-pass lesson for the code, and read the weight-loading and cache lessons carefully.
Do not know C? The C file is read-only for most readers; the setup lesson has a primer, and every lesson explains the result in words.
Only want performance? Go straight to Engine Fast after the forward pass.

#Exercise

You change the number of KV heads from 2 to 4 in Config. Which two things change and which file's behaviour do you expect to differ?

Answer: the KV cache becomes twice as large (more keys and values per token), and the attention reads more per step, so decode gets a little slower. The change appears in engine.py (the cache and the attention loop), and weight file sizes for the key and value matrices grow in gguf_mini.py outputs.

Next: set up, and a C primer.