Performance Techniques

Quantisation, speculative decoding, parallelism, disaggregated serving and caching, with the maths for each.

1. 1▶ Up nextPro17 min+50 XPQuantisation in depth: formats, groups, outliers, activations and KVHow quantisation actually works: scale and zero-point, per-tensor versus per-group scaling, why outliers hurt…
2. 2Pro15 min+50 XPSpeculative decoding in depth: acceptance maths, draft models, EAGLE and n-gramsHow speculative decoding trades idle compute for fewer weight reads: the exact acceptance rule that keeps outputs…
3. 3Pro15 min+50 XPModel parallelism: tensor, pipeline, expert and data parallel inferenceHow to split a model across GPUs: tensor parallelism and its per-layer all-reduce cost, pipeline parallelism and…
4. 4Pro13 min+50 XPDisaggregated prefill and decode: why and when to splitPrefill and decode interfere on shared GPUs. Chunked prefill versus disaggregation, KV-cache transfer costs, sizing…
5. 5Pro14 min+50 XPCaching: prefix caching, semantic caching and KV offloadThree different caches that are often confused: exact prefix KV reuse, semantic response caching, and tiered KV…