Models / Qwen3.5 35B-A3B
Best home rig for Qwen3.5 35B-A3B
Quick answer
How much VRAM does Qwen3.5 35B-A3B need?
The weights are about 22.0 GB at Q4_K_M (35.95B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime.
What is the fastest measured hardware for Qwen3.5 35B-A3B?
ThinkPad X1 Carbon Gen 13 (Core Ultra 7 258V, Arc 140V iGPU, 32 GB): Qwen3.5 35B-A3B IQ4_XS at 25.6 tok/s on llama.cpp Vulkan, 22.3 at 8K depth: 25.58 tok/s generation (llama.cpp 12377 Vulkan (Mesa ANV), IQ4_XS (16.95 GiB)), from https://github.com/ggml-org/llama.cpp/discussions/10879.
Last updated:
Legacy model35.95B total · 3B active4 measured resultsfastest plain decode: 25.58 tok/s model info ↗
Real, measured generation speeds for Qwen3.5 35B-A3B on home hardware, fastest first. Different quants/engines aren't directly comparable.
Prompt processing = prefill speed reported by the source, @ prompt length. * est. TTFT @8k is calculated from measured prefill, not measured. Why it matters.
Memory math: how much VRAM / RAM does Qwen3.5 35B-A3B need?
Qwen3.5 35B-A3B has 35.95B parameters, about 3B active per token (source: Qwen model card: 35B total / 3B activated MoE (8 routed + 1 shared experts), Apache 2.0, HF repo created 24 Feb 2026; HF safetensors total 35.95B (includes vision encoder)). Weight size = parameters × bits-per-weight ÷ 8, before context (KV cache) and runtime overhead.
| Format | Bits/weight | Weights | Smallest common memory size that holds the weights alone |
|---|---|---|---|
| Q8_0 | 8.50 | 38.2 GB | 48 GB |
| Q6_K | 6.56 | 29.5 GB | 32 GB |
| Q5_K_M | 5.70 | 25.6 GB | 32 GB |
| Q4_K_M | 4.89 | 22.0 GB | 24 GB |
| Q3_K_M | 4.00 | 18.0 GB | 24 GB |
| Q2_K | 3.16 | 14.2 GB | 16 GB |
Bits-per-weight for GGUF quants are llama.cpp's own figures (llama.cpp quantize README). "As measured" rows apply the source's stated bits-per-weight to the parameter count; real files differ slightly because some layers stay at higher precision. Try other combinations in the VRAM calculator.
Why it can still run on smaller GPUs: it is a mixture-of-experts model, so only about 3B of its 35.95B parameters are used for each generated token. Runtimes such as llama.cpp (--n-cpu-moe), ExLlamaV3 or Project Maya can keep most expert weights in system RAM and only the shared layers and hot experts in VRAM. Then you need VRAM + system RAM ≥ weights, and speed is limited mainly by RAM bandwidth.
FAQ
How much VRAM does Qwen3.5 35B-A3B need?
The weights are about 22.0 GB at Q4_K_M (35.95B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime.
What is the fastest measured hardware for Qwen3.5 35B-A3B?
ThinkPad X1 Carbon Gen 13 (Core Ultra 7 258V, Arc 140V iGPU, 32 GB): Qwen3.5 35B-A3B IQ4_XS at 25.6 tok/s on llama.cpp Vulkan, 22.3 at 8K depth: 25.58 tok/s generation (llama.cpp 12377 Vulkan (Mesa ANV), IQ4_XS (16.95 GiB)), from https://github.com/ggml-org/llama.cpp/discussions/10879.
Related: what GPU do you need for 2026 models · all builds