Ollama vs llama.cpp: which should you use?

People often ask whether to use Ollama or llama.cpp as if they were rivals. They are closer to a car and its engine: Ollama is built on llama.cpp. The real question is how much of the work you want done for you.

#What each one is

llama.cpp is the C and C++ inference engine that made running quantised models on ordinary hardware practical. It reads GGUF files and runs on CPUs, NVIDIA, AMD and Apple GPUs. It exposes a command-line tool and an OpenAI-compatible server, llama-server, and has a long list of flags for fine control.

Ollama wraps that engine in a friendly package: one installer, a model library you pull from by name, automatic GPU detection, a background service, and a simple local API.

#Starting each

bash

# Ollama
ollama run llama3.1:8b

# llama.cpp
llama-server -m ./Llama-3.1-8B-Instruct-Q4_K_M.gguf -ngl 99 -c 8192 --port 8080

Check each project's documentation for the exact current flags and model names.

#Where each is better

You want	Choose
The fastest path from nothing to a working model	Ollama
Models by simple names, with updates and storage handled	Ollama
Full control over layers on GPU, context size, batch sizes, cache types	llama.cpp
Newest features and quantisation types soonest	llama.cpp
A specific GGUF file from anywhere	Either (Ollama can import one)
Predictable behaviour in a production service you tune	llama.cpp, or a serving engine such as vLLM

#A reasonable path

Start with Ollama to learn what models feel like and to build a first application against its local API (Ollama quickstart).
Move to llama.cpp when you hit something Ollama does not expose, or you want to understand and tune performance (building llama.cpp).
Consider vLLM when many people use the model at once and throughput matters more than simplicity (vLLM for serving). On a Mac, also look at MLX (MLX).

#The mistakes to avoid

Assuming defaults are optimal. Check the log to confirm every layer is on the GPU. A model quietly left partly on the CPU is the most common reason for slow speed (tuning playbook).
Judging speed by feel. Measure tokens per second the same way each time. Bandwidth sets the ceiling (why).
Forgetting the context length. Both tools default to a modest context. Set what you need, and remember the KV cache grows with it (VRAM guide).

If you only remember one thing: Ollama for convenience, llama.cpp for control, and since they share an engine, what you learn about one carries over to the other. The runtime landscape lesson compares all the main options.