Best home rig for Llama 3.1 70B
Older model. For current picks see 2025–2026 models.
Real, measured generation speeds for Llama 3.1 70B on home hardware, fastest first. Different quants/engines aren't directly comparable.
| Build | Quant | Engine | tok/s | Cost | |
|---|---|---|---|---|---|
| Mac Studio M3 Ultra 512GB (Geerling test) | llama.cpp | 14.08 | source | ||
| Mac Studio M1 Ultra (64-core GPU) 128GB | llama.cpp | 9.84 | source | ||
| EPYC 7F52 jank rig: 4x3090 + 4090 + 4080 (136 GB VRAM) | Q8_0 | Oobabooga | 9 | source ⚠ unchecked | |
| Framework Desktop (AMD Strix Halo) mainboard | llama.cpp | 4.97 | source | ||
| Dell Pro Max with GB10 (Nvidia DGX Spark) | llama.cpp | 4.71 | source | ||
| AmpereOne A192-32X 512GB (CPU only) | llama.cpp | 3.86 | source | ||
| Ryzen 9 7900X + RTX 4090 | GPU+CPU offload | 3.1 | source |
SPONSOR
Your brand here. One sponsor slot on this page, shown to people choosing AI hardware. Sponsor RigBase →
Your brand here. One sponsor slot on this page, shown to people choosing AI hardware. Sponsor RigBase →
Rent before you buy: try this model on a cloud GPU for a few dollars first: RunPod
FAQ
What is the fastest measured hardware for Llama 3.1 70B?
Apple M3 Ultra: 14.08 tok/s generation (llama.cpp), from https://github.com/geerlingguy/ai-benchmarks/issues/42.
Related: what GPU do you need for 2026 models · all builds