Model runs on CPU instead of the GPU

Symptom

Likely causes

• llama.cpp built without a GPU backend
• -ngl is 0 (default for some builds)
• Model too large and Ollama silently spilled to CPU
• Driver or CUDA runtime mismatch

How to fix it

1. Run ollama ps and check the PROCESSOR column (it should read 100% GPU).
2. For llama.cpp, confirm startup logs say layers were offloaded (offloaded N/N layers to GPU).
3. Pass -ngl 99 and rebuild with -DGGML_CUDA=ON (NVIDIA) or use the Metal build on macOS.
4. Update GPU drivers; verify with nvidia-smi.
5. If memory is the cause, use a smaller quant or context.

ollama ps
llama-cli -m model.gguf -ngl 99 -p "hi" 2>&1 | grep -i offload