Quantisation in depth: formats, groups, outliers, activations and KV
How quantisation actually works: scale and zero-point, per-tensor versus per-group scaling, why outliers hurt, weight-only versus weight-activation quantisation, GPTQ/AWQ/GGUF/FP8 families, KV-cache quantisation, speed versus quality and how to evaluate. With a runnable error experiment.
17 min·Verified 2026-11Pro+50 XP
Quantisation stores numbers with fewer bits. For inference that means fewer bytes to move (faster decode), less memory (bigger models and batches fit) and sometimes faster arithmetic. The cost is error. This lesson explains how to get the benefit while controlling the error.
Pro content
Keep reading with Inference Academy Pro
Unlock the rest of this lesson, plus every advanced lesson, hands-on lab, the larger playground models and completion certificates.