Models / GLM-5.3 Flash
Best home rig for GLM-5.3 Flash
Current model320B total · 18B active5 measured resultsfastest plain decode: 105.3 tok/s model info ↗
Will it fit my GPU? Best value GPU per euro →
Real, measured generation speeds for GLM-5.3 Flash on home hardware, fastest first. Different quants/engines aren't directly comparable.
| Build | Quant | Engine | tok/s | Cost | Source |
|---|---|---|---|---|---|
| 4x DGX Spark ring running GLM-5.3 Flash NVFP4 | NVFP4 | vLLM TP4 (RiNGSiDE), single-stream code | 105.3 | github.com ↗ | |
| 4x DGX Spark ring running GLM-5.3 Flash NVFP4 | NVFP4 | vLLM TP4 (RiNGSiDE), single-stream prose | 61.7 | github.com ↗ | |
| 2x DGX Spark running GLM-5.3 Flash NVFP4 | NVFP4 | vLLM TP2 (RiNGSiDE), single-stream code | 56.5 | github.com ↗ | |
| 2x DGX Spark running GLM-5.3 Flash NVFP4 | NVFP4 | vLLM TP2 (RiNGSiDE), single-stream prose | 33.0 | github.com ↗ | |
| Ryzen 9 9950X3D + RTX 5090 + 128 GB DDR5: four big MoEs with experts offloaded to RAM (Qwen3.8 Flash-Next 78.8 tok/s) | EXL3 3.05 bpw, MTP drafting, experts offloaded to RAM | Project Maya 1.1.21 | 30.1 ⚡ | dev.to ↗ |
Memory math: how much VRAM / RAM does GLM-5.3 Flash need?
GLM-5.3 Flash has 320B parameters, about 18B active per token (source: Z.ai model card: 320B total, 18B active (Hugging Face lists 321B)). Weight size = parameters × bits-per-weight ÷ 8, before context (KV cache) and runtime overhead.
| Format | Bits/weight | Weights | Smallest common memory size that holds the weights alone |
|---|---|---|---|
| Q8_0 | 8.50 | 340.0 GB | 512 GB |
| Q6_K | 6.56 | 262.5 GB | 512 GB |
| Q5_K_M | 5.70 | 228.1 GB | 256 GB |
| Q4_K_M | 4.89 | 195.8 GB | 256 GB |
| Q3_K_M | 4.00 | 159.8 GB | 192 GB |
| Q2_K | 3.16 | 126.4 GB | 128 GB |
| As measured: EXL3 3.05 bpw, MTP drafting, experts offloaded to RAM | 3.05 | ≈122.0 GB | 128 GB |
Bits-per-weight for GGUF quants are llama.cpp's own figures (llama.cpp quantize README). "As measured" rows apply the source's stated bits-per-weight to the parameter count; real files differ slightly because some layers stay at higher precision. Try other combinations in the VRAM calculator.
Why it can still run on smaller GPUs: it is a mixture-of-experts model, so only about 18B of its 320B parameters are used for each generated token. Runtimes such as llama.cpp (--n-cpu-moe), ExLlamaV3 or Project Maya can keep most expert weights in system RAM and only the shared layers and hot experts in VRAM. Then you need VRAM + system RAM ≥ weights, and speed is limited mainly by RAM bandwidth.
Measured on setups with less GPU memory than the Q4_K_M weights (so part of the model lived in system RAM or was streamed from disk):
- RTX 5090 (32 GB) + 128 GB DDR5-5600 (4x32 GB): 30.1 tok/s (EXL3 3.05 bpw, MTP drafting, experts offloaded to RAM)
FAQ
How much VRAM does GLM-5.3 Flash need?
The weights are about 195.8 GB at Q4_K_M (320B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is 2x DGX Spark (GB10) (256 GB) at 56.5 tok/s.
What is the fastest measured hardware for GLM-5.3 Flash?
4x DGX Spark (GB10): 105.3 tok/s generation (vLLM TP4 (RiNGSiDE), single-stream code, NVFP4), from https://github.com/othexmr/GLM-5.3-Flash-NVFP4-2x-4x-DGX-Sparks-RiNGSiDE.
Which GPUs have measured results for GLM-5.3 Flash?
2x DGX Spark (GB10), 4x DGX Spark (GB10), RTX 5090.
Can it run on… 2x DGX Spark (GB10) · 4x DGX Spark (GB10) · RTX 5090
Related: what GPU do you need for 2026 models · all builds