Blog

Guides to running LLMs on your own hardware, with the numbers worked out. For structured, step-by-step teaching see the lessons.

• November 15, 2026 · 3 min read

  Can you run a 70B model at home? What hardware works

  A 70B model needs about 44 GiB at 4-bit. Which GPUs, Macs and CPUs can hold it, how fast each would be, and the cheapest realistic setups.

  hardwaresizing70b

• November 15, 2026 · 3 min read

  How much VRAM do you need to run an LLM?

  A worked method for sizing memory: weights, the KV cache and runtime overhead, with a Llama 3.1 8B example at every quantisation and a rule of thumb you can do in your head.

  vramsizingquantisation

• November 15, 2026 · 2 min read

  Ollama vs llama.cpp: which should you use?

  How the two relate (Ollama is built on llama.cpp), what each is good at, and a simple rule for choosing between them, with the commands to start.

  ollamallama.cppruntimes

• November 15, 2026 · 2 min read

  Which quantisation should you pick? Q4_K_M, Q5, Q8 and the rest

  What each GGUF quantisation level costs in size and quality, measured memory for a 70B model at every level, and a simple decision rule for choosing.

  quantisationggufsizing

• November 15, 2026 · 3 min read

  Why memory bandwidth, not compute, decides how fast an LLM runs

  Generating each token streams the whole model through memory. See the formula, then the predicted speed of Llama 3.1 8B across 14 GPUs, Macs and CPUs.

  performancebandwidthhardware

RSS feed