How much GPU do you need to fine-tune? Memory for full fine-tuning, LoRA and QLoRA
Where training memory goes (weights, gradients, optimiser state, activations), with a calculator and worked tables for 1B, 8B and 70B models: why full fine-tuning needs about 16 bytes per parameter, why LoRA and QLoRA cut that by 6 to 30 times, and how gradient checkpointing, sequence length and batch size move the total.
Fine-tuning fails at the same place for most people: an out-of-memory error. This lesson gives you the arithmetic to know before you start whether a job will fit, and what to change if it will not. The numbers are estimates from formulas you can run, not measurements of a particular library, so leave a margin and confirm with a short test run.
Pro content
Keep reading with Inference Academy Pro
Unlock the rest of this lesson, including 1 animated diagram, plus every advanced lesson, hands-on lab, the larger playground models and completion certificates.