Disaggregated prefill and decode: why and when to split
Prefill and decode interfere on shared GPUs. Chunked prefill versus disaggregation, KV-cache transfer costs, sizing prefill and decode pools, SLO-driven gains, and when the complexity is not worth it. With a worked sizing example.
13 min·Verified 2026-11Pro+50 XP
Prefill is compute-bound; decode is memory-bound. Put them on the same GPU and they fight. Disaggregation is the architectural response: separate machines for each phase.
Pro content
Keep reading with Inference Academy Pro
Unlock the rest of this lesson, plus every advanced lesson, hands-on lab, the larger playground models and completion certificates.