Attention optimisations: FlashAttention, GQA, paged KV and sliding windows
Why attention is memory-bound, how FlashAttention tiles to avoid materialising the score matrix, how MQA/GQA and MLA shrink the KV cache, how paged attention removes fragmentation, and when sliding windows or sparse patterns help. With KV-size formulas and worked examples.
16 min·Verified 2026-11Pro+50 XP
Attention is the part of a transformer whose cost grows with context length, and in serving it is where memory capacity and bandwidth are decided. Four ideas cover most of what modern engines do about it.
Pro content
Keep reading with Inference Academy Pro
Unlock the rest of this lesson, plus every advanced lesson, hands-on lab, the larger playground models and completion certificates.