GPU generations, CPU-GPU superchips, multi-GPU instances and MIG
What changed from Ampere to Hopper, Blackwell and Rubin for inference (memory, low-precision formats, interconnect), how Grace and Vera CPUs extend GPU memory, how to read multi-GPU instances and NVLink domains, when Multi-Instance GPU helps, and a worksheet that turns datasheet numbers into decode ceilings and capacity. Figures are approximate and must be verified.
Every new GPU generation comes with a launch slide full of big numbers. For inference only a few of them matter, and they matter for different reasons. This lesson gives you a way to read a generation: what it changes about memory, about low-precision formats and about how GPUs talk to each other. It builds on the GPU architecture and roofline lessons.
A warning that applies to the whole page: hardware figures change, early numbers get revised and many websites repeat marketing numbers (often "sparse" figures that are double the dense ones). The numbers below are approximate, taken from public reporting at the time of writing, and some belong to hardware that was still ramping. Verify against the vendor's datasheet and rely on dense, not sparse, figures.
Pro content
Keep reading with Inference Academy Pro
Unlock the rest of this lesson, including 1 animated diagram, plus every advanced lesson, hands-on lab, the larger playground models and completion certificates.