Models / Qwen3.6 35B-A3B / VRAM
How much VRAM do you need for Qwen3.6 35B-A3B?
Last updated:
Weight math from the model card, plus every measured RigBase result for Qwen3.6 35B-A3B. Leave extra room for context (KV cache).
Weight size by quant
| Quant | Bits/weight | Weights only |
|---|---|---|
| Q8_0 | 8.50 | 37.7 GB |
| Q6_K | 6.56 | 29.1 GB |
| Q5_K_M | 5.70 | 25.3 GB |
| Q4_K_M | 4.89 | 21.7 GB |
| Q3_K_M | 4.00 | 17.7 GB |
| Q2_K | 3.16 | 14.0 GB |
Bits-per-weight for GGUF rows: llama.cpp quantize README. Parameter count: GGUF param count via llamabench.ai; 3B active per name. We advise staying at Q4 or above for chat quality — see Is Q4 good enough?.
Measured speeds for Qwen3.6 35B-A3B
| Hardware | Quant | Engine | tok/s |
|---|---|---|---|
| Apple M5 Max (unified) · 128 GB | 4-bit MLX (mlx-serve build) | mlx-serve 26.10.2 + MTP | 271 ⚡ |
| RTX 5090 · 32 GB | llama-bench tg128 (quant per result) | llama.cpp | 258 |
| Apple M5 Max (unified) · 128 GB | 4-bit MLX | Rapid-MLX 0.16.1 + MTP | 240 ⚡ |
| 4x RTX 3090 · 96 GB | AWQ-INT4 + MTP-3 (TP=2) | vLLM | 225-231 ⚡ |
| Apple M5 Max (unified) · 36 GB | 4-bit MLX | Rapid-MLX 0.16.1 (MTP) | 173.4 ⚡ |
| RTX 4090 · 24 GB | Q4_K_M | llama.cpp b8931 | 160.6 |
| 4x RTX 3090 · 96 GB | AWQ-INT4 (TP=2) | vLLM | 149 |
| RTX 3090 · 24 GB | llama-bench tg128 (quant per result) | llama.cpp | 146-153 |
| Apple M5 Max (unified) · 36 GB | 4-bit MLX | oMLX 0.7.0 | 145.8 |
| RTX 3090 · 24 GB | Q4 GGUF, all layers on GPU, ~90K ctx | llama.cpp CUDA | 139.6 |
| AMD Radeon RX 7900 XTX · 24 GB | UD-Q4_K_XL + MTP (draft-n-max 4), ~16K context | llama.cpp ROCm (llama-server) | 135.2 ⚡ |
| Apple M5 Max (unified) · 36 GB | 4-bit MLX | LM Studio (MLX) | 124.5 |
⚡ = speculative decoding (MTP, DFlash…). Full list: best home rig for Qwen3.6 35B-A3B.
Related
All Qwen3.6 35B-A3B builds VRAM + KV calculator By VRAM size GGUF vs AWQ vs FP8 vs NVFP4 How much system RAM?
FAQ
How much VRAM do you need for Qwen3.6 35B-A3B?
Weights alone are about 21.7 GB at Q4_K_M (35.51B × 4.89 bits ÷ 8). Add memory for the KV cache (context) and the runtime. The smallest-memory measured result in RigBase is 24 GB (RTX 3090) at 146-153 tok/s. Going below Q4 usually hurts quality for chat; see the Q4 guide.
What is the fastest measured speed for Qwen3.6 35B-A3B?
RTX 5090: 258 tok/s plain decode (llama-bench tg128 (quant per result), llama.cpp).
Does system RAM replace VRAM?
Only for overflow. When the whole model fits in GPU or unified memory, system RAM barely matters for decode speed. See the system-RAM guide.
Referral link: if you sign up with Google and add $10, Runpod gives you and RigBase bonus credit (terms). It never affects our data.