Rollouts and model versioning
Treat models like releases: immutable bundles, quality gates, shadow traffic, canaries with real sample-size maths, progressive rollout, instant rollback, and the hidden things (templates, tokenizers, runtimes) that change behaviour.
A new quantization, a newer base model, a changed chat template, a tweaked system prompt or even a runtime upgrade can change outputs in ways your users notice. Ship these like software releases: versioned, tested, gradually rolled out and instantly reversible. The difference from ordinary software is that behaviour is statistical, so "tests pass" is not enough and "it looks fine" is not evidence.
Pro content
Keep reading with Inference Academy Pro
Unlock the rest of this lesson, plus every advanced lesson, hands-on lab, the larger playground models and completion certificates.