Speculative decoding in depth: acceptance maths, draft models, EAGLE and n-grams
How speculative decoding trades idle compute for fewer weight reads: the exact acceptance rule that keeps outputs unchanged, expected tokens per step and speedup formulas, choosing draft models, EAGLE/Medusa and n-gram variants, when it helps and when it hurts at high batch.
15 min·Verified 2026-11Pro+50 XP
Decoding reads all the weights for each token yet leaves the compute units mostly idle (roofline). Speculative decoding uses that idle compute: guess several tokens cheaply, then verify them all in one pass of the big model.
Pro content
Keep reading with Inference Academy Pro
Unlock the rest of this lesson, plus every advanced lesson, hands-on lab, the larger playground models and completion certificates.