Autoscaling policies in depth
Design autoscaling as a feedback control problem: target tracking maths, why delay causes oscillation, headroom from growth rate and cold start, step, scheduled and predictive scaling, combining signals, scale-in and draining, GPU node autoscaling, Kubernetes HPA and KEDA settings, and a simulated comparison of policies.
You can describe autoscaling in a sentence ("add replicas when busy, remove them when idle") and spend a year getting it right. The difficulty is that autoscaling is a feedback control system with a long delay: you observe load now, act now, and the effect arrives minutes later. Control systems with delay oscillate, overshoot or react too late unless designed carefully. This lesson gives you the design rules, the numbers and the Kubernetes settings.
Pro content
Keep reading with Inference Academy Pro
Unlock the rest of this lesson, plus every advanced lesson, hands-on lab, the larger playground models and completion certificates.