Models / DeepSeek V4.1 Flash
Best home rig for DeepSeek V4.1 Flash
Current model748.6B total · 16B active3 measured resultsfastest plain decode: 24.9 tok/s model info ↗
Will it fit my GPU? Best value GPU per euro →
Real, measured generation speeds for DeepSeek V4.1 Flash on home hardware, fastest first. Different quants/engines aren't directly comparable.
| Build | Quant | Engine | tok/s | Cost | Source |
|---|---|---|---|---|---|
| Mac Studio M3 Ultra 512 GB running DeepSeek V4.1 Flash with MTP | not stated | oMLX v0.7.1.dev1, MTP enabled | 30.8 ⚡ | omlx.ai ↗ | |
| Ryzen 9 9950X3D + RTX 5090 + 128 GB DDR5: four big MoEs with experts offloaded to RAM (Qwen3.8 Flash-Next 78.8 tok/s) | Maya-S (experts 2-3 bit), partly streamed from SSD | Project Maya 1.1.21 (maya-ds4) | 24.9 | dev.to ↗ | |
| Dual Tesla V100 32 GB on EPYC 7402 (bnk engine) | GSQ-RCO 2-bit | bnk v0.5.0, experts split VRAM/RAM (CPU offload) | 13 | github.com ↗ |
Memory math: how much VRAM / RAM does DeepSeek V4.1 Flash need?
DeepSeek V4.1 Flash has 748.6B parameters, about 16B active per token (source: vLLM recipe: 552B backbone + 196.6B Engram tables, 16B active per output token; official MXFP4/FP8 checkpoint ~511 GB). Weight size = parameters × bits-per-weight ÷ 8, before context (KV cache) and runtime overhead.
Per the vLLM recipe, 196.6B of the parameters are Engram lookup tables (about 183 GiB) that vLLM can keep in CPU memory, so the GPUs mainly need the 552B backbone. The native checkpoint mixes MXFP4 experts with FP8 weights.
| Format | Bits/weight | Weights | Smallest common memory size that holds the weights alone |
|---|---|---|---|
| Native release format | – | ≈511 GB | 512 GB |
| Q8_0 | 8.50 | 795.5 GB | 1024 GB |
| Q6_K | 6.56 | 614.2 GB | 1024 GB |
| Q5_K_M | 5.70 | 533.7 GB | 1024 GB |
| Q4_K_M | 4.89 | 458.0 GB | 512 GB |
| Q3_K_M | 4.00 | 373.9 GB | 512 GB |
| Q2_K | 3.16 | 295.6 GB | 512 GB |
Bits-per-weight for GGUF quants are llama.cpp's own figures (llama.cpp quantize README). "As measured" rows apply the source's stated bits-per-weight to the parameter count; real files differ slightly because some layers stay at higher precision. Try other combinations in the VRAM calculator.
Why it can still run on smaller GPUs: it is a mixture-of-experts model, so only about 16B of its 748.6B parameters are used for each generated token. Runtimes such as llama.cpp (--n-cpu-moe), ExLlamaV3 or Project Maya can keep most expert weights in system RAM and only the shared layers and hot experts in VRAM. Then you need VRAM + system RAM ≥ weights, and speed is limited mainly by RAM bandwidth.
Measured on setups with less GPU memory than the Q4_K_M / native weights (so part of the model lived in system RAM or was streamed from disk):
- RTX 5090 (32 GB) + 128 GB DDR5-5600 (4x32 GB): 24.9 tok/s (Maya-S (experts 2-3 bit), partly streamed from SSD)
- 2x Tesla V100 32 GB (PG500-216, sm_70) (64 GB) + 128 GB DDR4 (192 GB for DeepSeek): 13 tok/s (GSQ-RCO 2-bit)
FAQ
How much VRAM does DeepSeek V4.1 Flash need?
The weights are about 511 GB in its native format (vLLM recipe: 552B backbone + 196.6B Engram tables, 16B active per output token; official MXFP4/FP8 checkpoint ~511 GB). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is Apple M3 Ultra (512 GB) at 30.8 tok/s.
What is the fastest measured hardware for DeepSeek V4.1 Flash?
Apple M3 Ultra: 30.8 tok/s generation (oMLX v0.7.1.dev1, MTP enabled, not stated), from https://omlx.ai/benchmarks/performance/dvp1bn0s.
Which GPUs have measured results for DeepSeek V4.1 Flash?
2x Tesla V100 32GB, Apple M3 Ultra, RTX 5090.
Can it run on… 2x Tesla V100 32GB · Apple M3 Ultra · RTX 5090
Related: what GPU do you need for 2026 models · all builds