Tokens/second drops after a few minutes
Symptom
Fast at first, then noticeably slower during a long chat or sustained load.
Likely causes
- Thermal throttling (laptops, small cases)
- KV cache growth as context fills
- Memory pressure causing swap
How to fix it
- Check temperatures and clocks (nvidia-smi, powermetrics, or system monitors).
- Improve cooling, plug in, use a high-performance power mode.
- Benchmark at the context depth you actually use to separate cache growth from throttling.
- Quantize the KV cache or shorten history.
bash
nvidia-smi --query-gpu=temperature.gpu,clocks.sm,power.draw --format=csv -l 2Still stuck? Check the lessons or measure first with the benchmark lab.