AWQ, GPTQ, EXL2 and FP8: GPU-first formats
How GPU-oriented quantization works: GPTQ, AWQ, EXL2/EXL3 and FP8/INT8, group sizes and calibration, why weight-only quantization speeds decode but FP8 helps throughput, and how to choose and validate for your server.
22 min·Verified 2026-11Pro+50 XP
GGUF dominates CPU, Apple Silicon and mixed CPU/GPU setups. On NVIDIA GPU servers a different family of formats is common, built for GPU kernels, large batches and frameworks such as vLLM. This lesson explains how they work and how to choose between them.
Pro content
Keep reading with Inference Academy Pro
Unlock the rest of this lesson, plus every advanced lesson, hands-on lab, the larger playground models and completion certificates.