Lab: ship a monitored deployment
Stand up two model-server replicas behind an nginx gateway with Prometheus and Grafana, load-test it, find the concurrency where your SLO breaks, kill a replica on purpose, and make an alert fire.
In this lab you build the whole operational loop from this course: replicas, a gateway, metrics, dashboards, a load test, an alert and deliberate failures. By the end you will have a number, "the concurrency at which p95 TTFT crosses our SLO", and evidence that a replica failure causes a blip, not an outage.
Time: about an hour. You need: Docker with Compose, and a model: either a GPU with a 7 to 8B GGUF, or a small 1 to 3B GGUF on CPU (slower, but all the signals still appear). Use the load tester from concurrency and load testing.
Pro content
Keep reading with Inference Academy Pro
Unlock this full lab, plus every advanced lesson, hands-on lab, the larger playground models and completion certificates.