Know your workload: AI-native or add-on, online or offline, consumer or B2B, and what scale and specialisation change
Before choosing GPUs or engines, classify your application on four axes. See how interactive versus offline use changes cost per token by 6x on the same GPU, when self-hosting beats an API, why a small specialised model can cost a fraction of a general one, and how consumer and business traffic shapes differ. Numbers come from a short script.
Most serving mistakes are not technical. A team picks a model and a GPU, builds a service, and only later discovers that the workload was something else: far more batch than interactive, or far more specialised than general. This lesson is the step before capacity planning: classify the workload, because that classification sets almost every later decision.
Pro content
Keep reading with Inference Academy Pro
Unlock the rest of this lesson, including 1 animated diagram, plus every advanced lesson, hands-on lab, the larger playground models and completion certificates.