Best local LLMs for 8 GB of VRAM or unified memory

Measured speeds from builds where the model ran on up to 8 GB of GPU or unified memory. Each number links to its build and source.

8 GB · 12 GB · 16 GB · 24 GB · 32 GB · 48 GB · 64 GB · 96 GB · 128 GB or more

What fits by memory math

Q4_K_M weights (4.8944 bits per weight, per the llama.cpp quantize table) that fit in 8 GB with ~5% headroom, before KV cache: Mellum 2.1 12B-A2.5B ≈7 GB, Gemma 4 12B ≈7 GB, Qwen3.5 9B ≈5 GB, Llama 3.1 8B legacy ≈5 GB.

FAQ

Which local LLMs have been measured on 8 GB of VRAM?

No result in RigBase ran on up to 8 GB of memory yet. The memory-math list on this page shows what should fit.

What fits in 8 GB of VRAM?

By memory math at Q4_K_M (4.8944 bits per weight, llama.cpp), weights only, before KV cache and context: Mellum 2.1 12B-A2.5B (≈7 GB), Gemma 4 12B (≈7 GB), Qwen3.5 9B (≈5 GB), Llama 3.1 8B (≈5 GB). Leave a few GB for context.

Are speculative-decoding results comparable?

Not 1:1. MTP, DFlash and similar methods draft tokens that the main model verifies, which can double tok/s on predictable text. This page lists them in a separate column; tick the box to hide them in the full table.