Lab: benchmark your rig and compare with theory
Run llama-bench properly, compare your measurements with the bandwidth model, sweep offload, flash attention, context length and KV cache type, test for thermal throttling and write up where your machine is limited.
40 min·Verified 2026-11Hands-on labPro+100 XP
Goal: measure real prompt-processing and generation speed on your hardware, compare with what the bandwidth model predicts, find what limits your machine, and leave with a one-paragraph, shareable result and a reusable benchmark script.
Time: about 40 minutes. You need: llama.cpp installed (guide) and an 8B model at Q4_K_M as a baseline.
Pro content
Keep reading with Inference Academy Pro
Unlock this full lab, plus every advanced lesson, hands-on lab, the larger playground models and completion certificates.