Models & Quantization
GGUF, K-quants, AWQ, GPTQ, EXL2 and KV-cache quantization, and what each costs in quality.
0 of 5 complete
- ▶ Up next12 min+50 XPQuantization basicsHow storing weights in fewer bits shrinks and speeds up models: a worked numeric example, why outliers matter, how…
- 12 min+50 XPGGUF and the K-quant familyWhat is inside a GGUF file, how to read quant names like Q4_K_M and IQ3_M, what K-quants and importance matrices do…
- Pro22 min+50 XPAWQ, GPTQ, EXL2 and FP8: GPU-first formatsHow GPU-oriented quantization works: GPTQ, AWQ, EXL2/EXL3 and FP8/INT8, group sizes and calibration, why weight-only…
- Pro20 min+50 XPKV-cache quantizationShrink the cache that grows with every token of context: exact sizing for real models, q8_0 and q4_0 trade-offs, the…
- LabPro45 min+100 XPLab: choose a quant for your hardwareMeasure size, memory, speed, KL divergence and task quality for three quants of the same model, then make a…