Model parallelism: tensor, pipeline, expert and data parallel inference
How to split a model across GPUs: tensor parallelism and its per-layer all-reduce cost, pipeline parallelism and bubbles, expert parallelism for MoE, data parallel replicas, and how to choose with a communication-cost model and worked examples.
15 min·Verified 2026-11Pro+50 XP
If a model does not fit on one GPU, or one GPU is too slow for your latency target, you split it. How you split determines what communication you pay for and what you gain.
Pro content
Keep reading with Inference Academy Pro
Unlock the rest of this lesson, plus every advanced lesson, hands-on lab, the larger playground models and completion certificates.