Models / Qwen3.5 4B
Best home rig for Qwen3.5 4B
Quick answer
How much VRAM does Qwen3.5 4B need?
The weights are about 2.4 GB at Q4_K_M (4B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is Apple M4 (16 GB) at 29.7 tok/s.
What is the fastest measured hardware for Qwen3.5 4B?
RTX 2060 Super 8 GB + Core i9-10900K: Qwen3.5 9B Q4_K_M at 54.4 tok/s, Gemma 4 12B at 27.7 (llama.cpp): 83.7 tok/s generation (llama.cpp server (CUDA, Docker), Q4_K_M), from https://mustafa.net/2026/10/13/local-llm-calculator-tested-four-machines/.
Last updated:
Legacy model4B dense2 measured resultsfastest plain decode: 83.7 tok/s model info ↗
Older model. For current picks see current models.
Real, measured generation speeds for Qwen3.5 4B on home hardware, fastest first. Different quants/engines aren't directly comparable.
| Build | Quant | Engine | tok/s | Prompt processing (tok/s) | Cost | Source |
|---|---|---|---|---|---|---|
| RTX 2060 Super 8 GB + Core i9-10900K: Qwen3.5 9B Q4_K_M at 54.4 tok/s, Gemma 4 12B at 27.7 (llama.cpp) | Q4_K_M | llama.cpp server (CUDA, Docker) | 83.7 | – | mustafa.net ↗ | |
| Mac mini M4 (16 GB): Qwen3.5 9B Q4_K_M at 17.7 tok/s, Gemma 4 12B at 13.1 (llama.cpp Metal) | Q4_K_M | llama.cpp server (Metal) | 29.7 | – | mustafa.net ↗ |
Prompt processing = prefill speed reported by the source, @ prompt length. * est. TTFT @8k is calculated from measured prefill, not measured. Why it matters.
Memory math: how much VRAM / RAM does Qwen3.5 4B need?
Qwen3.5 4B has 4B parameters (source: Ollama library tag 4b (nominal)). Weight size = parameters × bits-per-weight ÷ 8, before context (KV cache) and runtime overhead.
| Format | Bits/weight | Weights | Smallest common memory size that holds the weights alone |
|---|---|---|---|
| Q8_0 | 8.50 | 4.3 GB | 8 GB |
| Q6_K | 6.56 | 3.3 GB | 8 GB |
| Q5_K_M | 5.70 | 2.9 GB | 8 GB |
| Q4_K_M | 4.89 | 2.4 GB | 8 GB |
| Q3_K_M | 4.00 | 2.0 GB | 8 GB |
| Q2_K | 3.16 | 1.6 GB | 8 GB |
Bits-per-weight for GGUF quants are llama.cpp's own figures (llama.cpp quantize README). "As measured" rows apply the source's stated bits-per-weight to the parameter count; real files differ slightly because some layers stay at higher precision. Try other combinations in the VRAM calculator.
FAQ
How much VRAM does Qwen3.5 4B need?
The weights are about 2.4 GB at Q4_K_M (4B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is Apple M4 (16 GB) at 29.7 tok/s.
What is the fastest measured hardware for Qwen3.5 4B?
RTX 2060 Super 8 GB + Core i9-10900K: Qwen3.5 9B Q4_K_M at 54.4 tok/s, Gemma 4 12B at 27.7 (llama.cpp): 83.7 tok/s generation (llama.cpp server (CUDA, Docker), Q4_K_M), from https://mustafa.net/2026/10/13/local-llm-calculator-tested-four-machines/.
Related: what GPU do you need for 2026 models · all builds