Models / Gemma 4 26B-A4B / VRAM
How much VRAM do you need for Gemma 4 26B-A4B?
Last updated:
Weight math from the model card, plus every measured RigBase result for Gemma 4 26B-A4B. Leave extra room for context (KV cache).
Weight size by quant
| Quant | Bits/weight | Weights only |
|---|---|---|
| Q8_0 | 8.50 | 26.8 GB |
| Q6_K | 6.56 | 20.7 GB |
| Q5_K_M | 5.70 | 18.0 GB |
| Q4_K_M | 4.89 | 15.4 GB |
| Q3_K_M | 4.00 | 12.6 GB |
| Q2_K | 3.16 | 10.0 GB |
Bits-per-weight for GGUF rows: llama.cpp quantize README. Parameter count: GGUF param count via llamabench.ai. We advise staying at Q4 or above for chat quality — see Is Q4 good enough?.
Measured speeds for Gemma 4 26B-A4B
| Hardware | Quant | Engine | tok/s |
|---|---|---|---|
| RTX 5090 · 32 GB | llama-bench tg128 (quant per result) | llama.cpp | 235 |
| RTX 5090 32GB · 32 GB | Q4_K_M (text-only build), f16 KV | llama-server b10448 | 232.4 |
| Apple M5 Max (unified) · 128 GB | 4-bit MLX (QAT) | Rapid-MLX 0.16.1 + DFlash | 166 ⚡ |
| RTX 4090 · 24 GB | llama-bench tg128 (quant per result) | llama.cpp | 164 |
| RX 9070 XT 16 GB + Radeon AI PRO R9700 32 GB · 48 GB | Q4 (UD-Q4_K_M per the repo KLD table) + MTP | llama.cpp (ROCm), layer split, MTP | 143.43 ⚡ |
| Intel Arc Pro B70 32 GB · 32 GB | UD-Q8_K_XL | llama.cpp SYCL (pinned build, oneAPI 202 | 122.16 |
| Nvidia GB10 (unified) · 128 GB | + official MTP drafter (k=3) | vLLM 0.31.0, 128K ctx, 1 user | 56.5 ⚡ |
| AMD Ryzen AI Max+ PRO 395 (Radeon 8060S iGPU, 32 GB dedicate · 128 GB | 4-bit | Ollama 0.34.0 | 51.8 |
| Apple M6 (12-core GPU, unified) · 32 GB | QAT (google/gemma-4-26b-a4b-qat) | LM Studio | 45.5 |
| Nvidia GB10 (unified) · 128 GB | no speculative decoding | vLLM 0.31.0, 128K ctx, 1 user | 30.8 |
⚡ = speculative decoding (MTP, DFlash…). Full list: best home rig for Gemma 4 26B-A4B.
Related
All Gemma 4 26B-A4B builds VRAM + KV calculator By VRAM size GGUF vs AWQ vs FP8 vs NVFP4 How much system RAM?
FAQ
How much VRAM do you need for Gemma 4 26B-A4B?
Weights alone are about 15.4 GB at Q4_K_M (25.23B × 4.89 bits ÷ 8). Add memory for the KV cache (context) and the runtime. The smallest-memory measured result in RigBase is 24 GB (RTX 4090) at 164 tok/s. Going below Q4 usually hurts quality for chat; see the Q4 guide.
What is the fastest measured speed for Gemma 4 26B-A4B?
RTX 5090: 235 tok/s plain decode (llama-bench tg128 (quant per result), llama.cpp).
Does system RAM replace VRAM?
Only for overflow. When the whole model fits in GPU or unified memory, system RAM barely matters for decode speed. See the system-RAM guide.
Referral link: if you sign up with Google and add $10, Runpod gives you and RigBase bonus credit (terms). It never affects our data.