vLLM fails at startup with an out-of-memory or KV cache error
Symptom
No available memory for the cache blocks or the model needs more KV memory than available at the requested --max-model-len.
Likely causes
- --max-model-len too large for the free memory
- --gpu-memory-utilization too low or the GPU is shared
- Model too large without quantization
How to fix it
- Lower --max-model-len and --max-num-seqs.
- Raise --gpu-memory-utilization (e.g. 0.92) if the GPU is dedicated.
- Use an AWQ/GPTQ/FP8 model or --kv-cache-dtype fp8.
- Use --tensor-parallel-size across more GPUs.
bash
vllm serve MODEL --max-model-len 8192 --gpu-memory-utilization 0.92 --kv-cache-dtype fp8Still stuck? Check the lessons or measure first with the benchmark lab.