Can I run this model on my hardware?
Choose a model, then your GPU or Mac. Each page shows the memory needed at every quantisation, the longest context that fits and an estimated speed. For custom settings use the VRAM calculator.
Llama 3.2 1B
- NVIDIA RTX 3060 12GB
- NVIDIA RTX 4060 Ti 16GB
- NVIDIA RTX 4070 12GB
- NVIDIA RTX 3090 24GB
- NVIDIA RTX 4090 24GB
- NVIDIA RTX 5090 32GB
- AMD RX 7900 XTX 24GB
- Apple M2 (16GB)
- Apple M3 Pro (36GB)
- Apple M4 Pro (48GB)
- Apple M3 Max (64GB)
- Apple M4 Max (128GB)
- Apple M3 Ultra (192GB)
- CPU only, dual-channel DDR5 (32GB)
- CPU only, dual-channel DDR4 (16GB)
Llama 3.2 3B
- NVIDIA RTX 3060 12GB
- NVIDIA RTX 4060 Ti 16GB
- NVIDIA RTX 4070 12GB
- NVIDIA RTX 3090 24GB
- NVIDIA RTX 4090 24GB
- NVIDIA RTX 5090 32GB
- AMD RX 7900 XTX 24GB
- Apple M2 (16GB)
- Apple M3 Pro (36GB)
- Apple M4 Pro (48GB)
- Apple M3 Max (64GB)
- Apple M4 Max (128GB)
- Apple M3 Ultra (192GB)
- CPU only, dual-channel DDR5 (32GB)
- CPU only, dual-channel DDR4 (16GB)
Phi-3 mini 3.8B
- NVIDIA RTX 3060 12GB
- NVIDIA RTX 4060 Ti 16GB
- NVIDIA RTX 4070 12GB
- NVIDIA RTX 3090 24GB
- NVIDIA RTX 4090 24GB
- NVIDIA RTX 5090 32GB
- AMD RX 7900 XTX 24GB
- Apple M2 (16GB)
- Apple M3 Pro (36GB)
- Apple M4 Pro (48GB)
- Apple M3 Max (64GB)
- Apple M4 Max (128GB)
- Apple M3 Ultra (192GB)
- CPU only, dual-channel DDR5 (32GB)
- CPU only, dual-channel DDR4 (16GB)
Qwen2.5 7B
- NVIDIA RTX 3060 12GB
- NVIDIA RTX 4060 Ti 16GB
- NVIDIA RTX 4070 12GB
- NVIDIA RTX 3090 24GB
- NVIDIA RTX 4090 24GB
- NVIDIA RTX 5090 32GB
- AMD RX 7900 XTX 24GB
- Apple M2 (16GB)
- Apple M3 Pro (36GB)
- Apple M4 Pro (48GB)
- Apple M3 Max (64GB)
- Apple M4 Max (128GB)
- Apple M3 Ultra (192GB)
- CPU only, dual-channel DDR5 (32GB)
- CPU only, dual-channel DDR4 (16GB)
Mistral 7B v0.3
- NVIDIA RTX 3060 12GB
- NVIDIA RTX 4060 Ti 16GB
- NVIDIA RTX 4070 12GB
- NVIDIA RTX 3090 24GB
- NVIDIA RTX 4090 24GB
- NVIDIA RTX 5090 32GB
- AMD RX 7900 XTX 24GB
- Apple M2 (16GB)
- Apple M3 Pro (36GB)
- Apple M4 Pro (48GB)
- Apple M3 Max (64GB)
- Apple M4 Max (128GB)
- Apple M3 Ultra (192GB)
- CPU only, dual-channel DDR5 (32GB)
- CPU only, dual-channel DDR4 (16GB)
Llama 3.1 8B
- NVIDIA RTX 3060 12GB
- NVIDIA RTX 4060 Ti 16GB
- NVIDIA RTX 4070 12GB
- NVIDIA RTX 3090 24GB
- NVIDIA RTX 4090 24GB
- NVIDIA RTX 5090 32GB
- AMD RX 7900 XTX 24GB
- Apple M2 (16GB)
- Apple M3 Pro (36GB)
- Apple M4 Pro (48GB)
- Apple M3 Max (64GB)
- Apple M4 Max (128GB)
- Apple M3 Ultra (192GB)
- CPU only, dual-channel DDR5 (32GB)
- CPU only, dual-channel DDR4 (16GB)
Gemma 2 9B
- NVIDIA RTX 3060 12GB
- NVIDIA RTX 4060 Ti 16GB
- NVIDIA RTX 4070 12GB
- NVIDIA RTX 3090 24GB
- NVIDIA RTX 4090 24GB
- NVIDIA RTX 5090 32GB
- AMD RX 7900 XTX 24GB
- Apple M2 (16GB)
- Apple M3 Pro (36GB)
- Apple M4 Pro (48GB)
- Apple M3 Max (64GB)
- Apple M4 Max (128GB)
- Apple M3 Ultra (192GB)
- CPU only, dual-channel DDR5 (32GB)
- CPU only, dual-channel DDR4 (16GB)
Qwen2.5 14B
- NVIDIA RTX 3060 12GB
- NVIDIA RTX 4060 Ti 16GB
- NVIDIA RTX 4070 12GB
- NVIDIA RTX 3090 24GB
- NVIDIA RTX 4090 24GB
- NVIDIA RTX 5090 32GB
- AMD RX 7900 XTX 24GB
- Apple M2 (16GB)
- Apple M3 Pro (36GB)
- Apple M4 Pro (48GB)
- Apple M3 Max (64GB)
- Apple M4 Max (128GB)
- Apple M3 Ultra (192GB)
- CPU only, dual-channel DDR5 (32GB)
- CPU only, dual-channel DDR4 (16GB)
Qwen2.5 32B
- NVIDIA RTX 3060 12GB
- NVIDIA RTX 4060 Ti 16GB
- NVIDIA RTX 4070 12GB
- NVIDIA RTX 3090 24GB
- NVIDIA RTX 4090 24GB
- NVIDIA RTX 5090 32GB
- AMD RX 7900 XTX 24GB
- Apple M2 (16GB)
- Apple M3 Pro (36GB)
- Apple M4 Pro (48GB)
- Apple M3 Max (64GB)
- Apple M4 Max (128GB)
- Apple M3 Ultra (192GB)
- CPU only, dual-channel DDR5 (32GB)
- CPU only, dual-channel DDR4 (16GB)
Mixtral 8x7B (MoE)
- NVIDIA RTX 3060 12GB
- NVIDIA RTX 4060 Ti 16GB
- NVIDIA RTX 4070 12GB
- NVIDIA RTX 3090 24GB
- NVIDIA RTX 4090 24GB
- NVIDIA RTX 5090 32GB
- AMD RX 7900 XTX 24GB
- Apple M2 (16GB)
- Apple M3 Pro (36GB)
- Apple M4 Pro (48GB)
- Apple M3 Max (64GB)
- Apple M4 Max (128GB)
- Apple M3 Ultra (192GB)
- CPU only, dual-channel DDR5 (32GB)
- CPU only, dual-channel DDR4 (16GB)
Llama 3.1 70B
- NVIDIA RTX 3060 12GB
- NVIDIA RTX 4060 Ti 16GB
- NVIDIA RTX 4070 12GB
- NVIDIA RTX 3090 24GB
- NVIDIA RTX 4090 24GB
- NVIDIA RTX 5090 32GB
- AMD RX 7900 XTX 24GB
- Apple M2 (16GB)
- Apple M3 Pro (36GB)
- Apple M4 Pro (48GB)
- Apple M3 Max (64GB)
- Apple M4 Max (128GB)
- Apple M3 Ultra (192GB)
- CPU only, dual-channel DDR5 (32GB)
- CPU only, dual-channel DDR4 (16GB)