<!-- Markdown mirror of https://getrigbase.com/models/qwen3-5-397b-a17b/ . Source of truth is the HTML page. Data licence: CC BY 4.0, please cite RigBase (https://getrigbase.com/models/qwen3-5-397b-a17b/). -->
Source: https://getrigbase.com/models/qwen3-5-397b-a17b/
Last updated: 2026-11-17

[Models](https://getrigbase.com/models/) / Qwen3.5 397B-A17B

# Best home rig for Qwen3.5 397B-A17B

Quick answer

**How much VRAM does Qwen3.5 397B-A17B need?**
The weights are about 246.6 GB at Q4_K_M (403B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is Apple M5 Ultra (256 GB) at 61.6 tok/s.

**What is the fastest measured hardware for Qwen3.5 397B-A17B?**
Apple M5 Ultra: 61.6 tok/s generation (mlx-lm 0.32.2 (MLX 0.32.4), 4-bit MLX (mlx-community, ~224 GB)), from https://bohemiansilicon.com/blog/qwen3-5-397b-mac-studio-m5-ultra-256gb/.

Last updated: 17 Nov 2026

Legacy model 403B total · 17B active 1 measured result fastest plain decode: 61.6 tok/s  [model info ↗](https://huggingface.co/Qwen/Qwen3.5-397B-A17B)

**Newer model available: [Qwen3.8 Flash-Next](https://getrigbase.com/models/qwen3-8-flash-next/)**. Qwen3.5 397B-A17B is a legacy model; its numbers stay here for reference. Qwen’s current large open MoE. [source ↗](https://www.osbbd.com/open-models)

Real, measured generation speeds for Qwen3.5 397B-A17B on home hardware, fastest first. Different quants/engines aren't directly comparable.

| Build | Quant | Engine | tok/s | Prompt processing (tok/s) | Cost | Source |
|---|---|---|---|---|---|---|
| [Mac Studio M5 Ultra 256 GB: Qwen3.5 397B-A17B 4-bit MLX (224 GB) at 61.6 tok/s, 44.1 at 131K context (Bohemian Silicon)](https://getrigbase.com/builds/mac-studio-m5-ultra-256gb-qwen35-397b-a17b-mlx-bohemiansilicon/) | 4-bit MLX (mlx-community, ~224 GB) | mlx-lm 0.32.2 (MLX 0.32.4) | **61.6** | 1,265 @8K · est. TTFT @8k ≈6.5 s* |  | [bohemiansilicon.com ↗](https://bohemiansilicon.com/blog/qwen3-5-397b-mac-studio-m5-ultra-256gb/) |

Prompt processing = prefill speed reported by the source, @ prompt length. * est. TTFT @8k is calculated from measured prefill, not measured. [Why it matters](https://getrigbase.com/can-it-run/#prefill).
Ad · this space is available **Sponsor this spot: reach people building local AI rigs.** One clearly-labeled slot per page, no tracking scripts.  [Sponsor this spot →](https://getrigbase.com/advertise/#inquiry)

**Rent before you buy:** try this model on a cloud GPU for a few dollars first: [Runpod](https://runpod.io?ref=77qcpl2c) (referral link; sign up with Google to get the new-user bonus)

## Memory math: how much VRAM / RAM does Qwen3.5 397B-A17B need?

Qwen3.5 397B-A17B has **403B parameters**, about **17B active** per token ([source](https://huggingface.co/Qwen/Qwen3.5-397B-A17B): Qwen Hugging Face repo (Feb 2026): 397B total / ~17B active MoE; HF API lists 403.4B params in the safetensors (vision tower included)). Weight size = parameters × bits-per-weight ÷ 8, before context (KV cache) and runtime overhead.

| Format | Bits/weight | Weights | Smallest common memory size that holds the weights alone |
|---|---|---|---|
| Q8_0 | 8.50 | 428.2 GB | 512 GB |
| Q6_K | 6.56 | 330.6 GB | 512 GB |
| Q5_K_M | 5.70 | 287.3 GB | 512 GB |
| Q4_K_M | 4.89 | **246.6 GB** | 256 GB |
| Q3_K_M | 4.00 | 201.3 GB | 256 GB |
| Q2_K | 3.16 | 159.1 GB | 192 GB |

Bits-per-weight for GGUF quants are llama.cpp's own figures ([llama.cpp quantize README](https://github.com/ggml-org/llama.cpp/blob/master/tools/quantize/README.md)). "As measured" rows apply the source's stated bits-per-weight to the parameter count; real files differ slightly because some layers stay at higher precision. Try other combinations in the [VRAM calculator](https://getrigbase.com/can-it-run/).

**Why it can still run on smaller GPUs:** it is a mixture-of-experts model, so only about 17B of its 403B parameters are used for each generated token. Runtimes such as llama.cpp (--n-cpu-moe), ExLlamaV3 or Project Maya can keep most expert weights in system RAM and only the shared layers and hot experts in VRAM. Then you need **VRAM + system RAM ≥ weights**, and speed is limited mainly by RAM bandwidth.

## FAQ

### How much VRAM does Qwen3.5 397B-A17B need?

The weights are about 246.6 GB at Q4_K_M (403B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is Apple M5 Ultra (256 GB) at 61.6 tok/s.

### What is the fastest measured hardware for Qwen3.5 397B-A17B?

Apple M5 Ultra: 61.6 tok/s generation (mlx-lm 0.32.2 (MLX 0.32.4), 4-bit MLX (mlx-community, ~224 GB)), from https://bohemiansilicon.com/blog/qwen3-5-397b-mac-studio-m5-ultra-256gb/.

Related: [what GPU do you need for 2026 models](https://getrigbase.com/guides/what-gpu-for-qwen3-6-gemma-4-gpt-oss/) · [all builds](https://getrigbase.com/builds/)
