Rent before you buy: try this model on a cloud GPU for a few dollars first: Runpod(referral link; sign up with Google to get the new-user bonus)
Memory math: how much VRAM / RAM does Qwen3.8 Flash-Next need?
Qwen3.8 Flash-Next has 180B parameters, about 6B active per token (source: ~180B total per the model card as summarised by strata→signal (125B main network, ~6B active per token, 51B lookup table, 4B draft head)). Weight size = parameters × bits-per-weight ÷ 8, before context (KV cache) and runtime overhead.
Format
Bits/weight
Weights
Smallest common memory size that holds the weights alone
Q8_0
8.50
191.3 GB
192 GB
Q6_K
6.56
147.7 GB
192 GB
Q5_K_M
5.70
128.3 GB
192 GB
Q4_K_M
4.89
110.1 GB
128 GB
Q3_K_M
4.00
89.9 GB
96 GB
Q2_K
3.16
71.1 GB
96 GB
Bits-per-weight for GGUF quants are llama.cpp's own figures (llama.cpp quantize README). "As measured" rows apply the source's stated bits-per-weight to the parameter count; real files differ slightly because some layers stay at higher precision. Try other combinations in the VRAM calculator.
Why it can still run on smaller GPUs: it is a mixture-of-experts model, so only about 6B of its 180B parameters are used for each generated token. Runtimes such as llama.cpp (--n-cpu-moe), ExLlamaV3 or Project Maya can keep most expert weights in system RAM and only the shared layers and hot experts in VRAM. Then you need VRAM + system RAM ≥ weights, and speed is limited mainly by RAM bandwidth.
Measured on setups with less GPU memory than the Q4_K_M weights (so part of the model lived in system RAM or was streamed from disk):
The weights are about 110.1 GB at Q4_K_M (180B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is DGX Spark (GB10) (128 GB) at 26.8 tok/s.
What is the fastest measured hardware for Qwen3.8 Flash-Next?
EPYC 7532 workstation (RTX Pro 6000 + RTX 3090 Ti, 300 W caps): Qwen3.8-Flash-Next, llama.cpp vs Strata engine: 205.263 tok/s generation (Strata engine 0.1.39, RTX Pro 6000, IQ3_S + model draft head, all experts on the RTX Pro 6000), from https://research.strata2signal.com/flash-next-on-three-cards/.
Which GPUs have measured results for Qwen3.8 Flash-Next?
2x Intel Arc Pro B70, 2x RTX 5070 Ti, 2x RTX 5080, 2x RTX PRO 4000 Blackwell, 2x Tesla V100 32GB, DGX Spark (GB10), RTX 5090.