Models / Gemma 4 31B / VRAM
How much VRAM do you need for Gemma 4 31B?
Last updated:
Weight math from the model card, plus every measured RigBase result for Gemma 4 31B. Leave extra room for context (KV cache).
Weight size by quant
| Quant | Bits/weight | Weights only |
|---|---|---|
| Q8_0 | 8.50 | 32.6 GB |
| Q6_K | 6.56 | 25.2 GB |
| Q5_K_M | 5.70 | 21.9 GB |
| Q4_K_M | 4.89 | 18.8 GB |
| Q3_K_M | 4.00 | 15.3 GB |
| Q2_K | 3.16 | 12.1 GB |
Bits-per-weight for GGUF rows: llama.cpp quantize README. Parameter count: GGUF param count via llamabench.ai. We advise staying at Q4 or above for chat quality — see Is Q4 good enough?.
Measured speeds for Gemma 4 31B
| Hardware | Quant | Engine | tok/s |
|---|---|---|---|
| RTX 5090 · 32 GB | Q4_K_M | llama.cpp b11658 (llama-bench) | 71.3 |
| RTX 5090 · 32 GB | llama-bench tg128 (quant per result) | llama.cpp | 70.0 |
| RX 9070 XT 16 GB + Radeon AI PRO R9700 32 GB · 48 GB | Q6_K + MTP drafter, f16 KV | llama.cpp b11434 (ROCm), layer split, MT | 52.2 ⚡ |
| RTX 4090 · 24 GB | llama-bench tg128 (quant per result) | llama.cpp | 43.8 |
| RTX 3090 · 24 GB | Q4_K_M | llama.cpp b11658 (llama-bench) | 38.8 |
| RX 9070 XT 16 GB + Radeon AI PRO R9700 32 GB · 48 GB | Q6_K, f16 KV | llama.cpp b11434 (ROCm), layer split, no | 19.3 |
| AMD Ryzen AI Max+ PRO 395 (Radeon 8060S iGPU, 32 GB dedicate · 128 GB | 4-bit | Ollama 0.34.0 | 8.8 |
| Apple M6 (12-core GPU, unified) · 32 GB | QAT (google/gemma-4-31b-qat) | LM Studio | 7.4 |
⚡ = speculative decoding (MTP, DFlash…). Full list: best home rig for Gemma 4 31B.
Related
All Gemma 4 31B builds VRAM + KV calculator By VRAM size GGUF vs AWQ vs FP8 vs NVFP4 How much system RAM?
FAQ
How much VRAM do you need for Gemma 4 31B?
Weights alone are about 18.8 GB at Q4_K_M (30.7B × 4.89 bits ÷ 8). Add memory for the KV cache (context) and the runtime. The smallest-memory measured result in RigBase is 24 GB (RTX 4090) at 43.8 tok/s. Going below Q4 usually hurts quality for chat; see the Q4 guide.
What is the fastest measured speed for Gemma 4 31B?
RTX 5090: 71.3 tok/s plain decode (Q4_K_M, llama.cpp b11658 (llama-bench)).
Does system RAM replace VRAM?
Only for overflow. When the whole model fits in GPU or unified memory, system RAM barely matters for decode speed. See the system-RAM guide.
Referral link: if you sign up with Google and add $10, Runpod gives you and RigBase bonus credit (terms). It never affects our data.