Models / Flint 40B-A2.5B / VRAM
How much VRAM do you need for Flint 40B-A2.5B?
Last updated:
Weight math from the model card, plus every measured RigBase result for Flint 40B-A2.5B. Leave extra room for context (KV cache).
Weight size by quant
| Quant | Bits/weight | Weights only |
|---|---|---|
| Q8_0 | 8.50 | 42.3 GB |
| Q6_K | 6.56 | 32.7 GB |
| Q5_K_M | 5.70 | 28.4 GB |
| Q4_K_M | 4.89 | 24.3 GB |
| Q3_K_M | 4.00 | 19.9 GB |
| Q2_K | 3.16 | 15.7 GB |
Bits-per-weight for GGUF rows: llama.cpp quantize README. Parameter count: 39.8B total / ~2.5B active MoE coding model (Recursive, Apache 2.0), per Hugging. We advise staying at Q4 or above for chat quality — see Is Q4 good enough?.
Measured speeds for Flint 40B-A2.5B
| Hardware | Quant | Engine | tok/s |
|---|---|---|---|
| RTX 5090 32GB · 32 GB | NVFP4 GGUF (RecursiveSI/flint-nvfp4-gguf), f16 KV | llama.cpp b11803, no speculation | 224.9 |
| RTX 4090 · 24 GB | NVFP4 | llama.cpp 0.7.0-dev (build 11803) | 179.4 |
| RTX 4090 · 24 GB | UD-Q3_K_XL | llama.cpp 0.7.0-dev (build 11803) | 169.2 |
| RTX 3090 · 24 GB | UD-Q3_K_XL | llama.cpp 0.7.0-dev (build 11803) | 128.9 |
| Apple M4 Pro (20-core GPU, unified) · 48 GB | int4 MLX (RecursiveSI/flint-int4) | Kapitsa (PR #286) | 127.9 |
| Apple M4 Pro (20-core GPU, unified) · 48 GB | int4 MLX (same files) | rec 0.1.0 (Recursive) | 126.56 |
| RTX 5090 32GB · 32 GB | NVFP4 GGUF, f16 KV | llama.cpp b11803, no speculation | 110.2 |
| Apple M4 Pro (20-core GPU, unified) · 48 GB | Q4_1 GGUF (RecursiveSI/flint-int4-gguf) | llama.cpp b11804, all layers on GPU, -fa | 79.57 |
⚡ = speculative decoding (MTP, DFlash…). Full list: best home rig for Flint 40B-A2.5B.
Related
All Flint 40B-A2.5B builds VRAM + KV calculator By VRAM size GGUF vs AWQ vs FP8 vs NVFP4 How much system RAM?
FAQ
How much VRAM do you need for Flint 40B-A2.5B?
Weights alone are about 24.3 GB at Q4_K_M (39.8B × 4.89 bits ÷ 8). Add memory for the KV cache (context) and the runtime. The smallest-memory measured result in RigBase is 24 GB (RTX 4090) at 179.4 tok/s. Going below Q4 usually hurts quality for chat; see the Q4 guide.
What is the fastest measured speed for Flint 40B-A2.5B?
RTX 5090 32GB: 224.9 tok/s plain decode (NVFP4 GGUF (RecursiveSI/flint-nvfp4-gguf), f16 KV, llama.cpp b11803, no speculation).
Does system RAM replace VRAM?
Only for overflow. When the whole model fits in GPU or unified memory, system RAM barely matters for decode speed. See the system-RAM guide.
Referral link: if you sign up with Google and add $10, Runpod gives you and RigBase bonus credit (terms). It never affects our data.