Accelerators beyond NVIDIA: AMD, TPU, Trainium, Gaudi and SRAM-based chips compared
What actually differs between inference chips (memory design, matrix units, interconnect, software stack and how you buy them), a worksheet that turns spec-sheet numbers and a software-efficiency factor into tokens per second and cost per million tokens, and a checklist for evaluating a non-NVIDIA option without being fooled by peak numbers. Uses hypothetical chips, not vendor claims.
For most people, "inference hardware" means NVIDIA GPUs. That is a reasonable default, because the software is the most mature. But other accelerators exist, some are cheaper or easier to rent, and a few are much faster for specific workloads. This lesson gives you a way to compare them using the same ideas you already know from the roofline and GPU architecture lessons.
Chip lines, product names, specs and prices change fast. Everything below is conceptual, and every number in the worksheet belongs to hypothetical chips. Plug in current datasheet values and your own measurements before deciding anything.
Pro content
Keep reading with Inference Academy Pro
Unlock the rest of this lesson, including 1 animated diagram, plus every advanced lesson, hands-on lab, the larger playground models and completion certificates.