Command builder

Build correct llama.cpp, Ollama, vLLM and MLX launch commands and learn what every flag does.

EngineModel (HF repo:quant, or file path)

Context / slotParallel users

GPU offload / GPU countllama.cpp: layers. vLLM: number of GPUs.

KV cacheFlash attentionPortCPU threads

Listen on all interfaces Require API key

shell

llama-server \
  -hf bartowski/Meta-Llama-3.1-8B-Instruct-GGUF:Q4_K_M \
  -c 8192 \
  -ngl 99 \
  --flash-attn on \
  --port 8080 \
  --jinja

-hf bartowski/Meta-Llama-3.1-8B-Instruct-GGUF:Q4_K_M: Download/cache the model from Hugging Face (repo:quant).
-c 8192: Context window in tokens.
-ngl 99: Offload all layers to the GPU (use a lower number if you run out of VRAM).
--flash-attn on: Flash attention: lower memory, faster long context. Older builds use bare -fa.
--port 8080: HTTP port; API at /v1, web UI at /.
--jinja: Use the model's own chat/tool template.

Flags change between releases. Run --help on your version, and see the runtime lessons.

Learn more