Models / LFM2.5 24B-A2B / VRAM
How much VRAM do you need for LFM2.5 24B-A2B?
Last updated:
Weight math from the model card, plus every measured RigBase result for LFM2.5 24B-A2B. Leave extra room for context (KV cache).
Weight size by quant
| Quant | Bits/weight | Weights only |
|---|---|---|
| Q8_0 | 8.50 | 25.5 GB |
| Q6_K | 6.56 | 19.7 GB |
| Q5_K_M | 5.70 | 17.1 GB |
| Q4_K_M | 4.89 | 14.7 GB |
| Q3_K_M | 4.00 | 12.0 GB |
| Q2_K | 3.16 | 9.5 GB |
Bits-per-weight for GGUF rows: llama.cpp quantize README. Parameter count: via llamabench.ai. We advise staying at Q4 or above for chat quality — see Is Q4 good enough?.
Measured speeds for LFM2.5 24B-A2B
| Hardware | Quant | Engine | tok/s |
|---|---|---|---|
| RTX 4090 · 24 GB | llama-bench tg128 (quant per result) | llama.cpp | 261-339 |
| NVIDIA TITAN RTX · 24 GB | Unsloth UD-Q4_K_XL | llama.cpp b11222 (CUDA) | 190.7 |
| RTX 4060 Ti 16GB · 16 GB | llama-bench tg128 (quant per result) | llama.cpp | 131-150 |
| RTX 3060 · 12 GB | llama-bench tg128 (quant per result) | llama.cpp | 125 |
| AMD Radeon 8060S (Strix Halo, unified) · 128 GB | Unsloth UD-Q4_K_XL (13.9 GiB) | llama.cpp (Vulkan) | 100.2 |
⚡ = speculative decoding (MTP, DFlash…). Full list: best home rig for LFM2.5 24B-A2B.
Related
All LFM2.5 24B-A2B builds VRAM + KV calculator By VRAM size GGUF vs AWQ vs FP8 vs NVFP4 How much system RAM?
FAQ
How much VRAM do you need for LFM2.5 24B-A2B?
Weights alone are about 14.7 GB at Q4_K_M (23.97B × 4.89 bits ÷ 8). Add memory for the KV cache (context) and the runtime. The smallest-memory measured result in RigBase is 12 GB (RTX 3060) at 125 tok/s. Going below Q4 usually hurts quality for chat; see the Q4 guide.
What is the fastest measured speed for LFM2.5 24B-A2B?
RTX 4090: 261-339 tok/s plain decode (llama-bench tg128 (quant per result), llama.cpp).
Does system RAM replace VRAM?
Only for overflow. When the whole model fits in GPU or unified memory, system RAM barely matters for decode speed. See the system-RAM guide.
Referral link: if you sign up with Google and add $10, Runpod gives you and RigBase bonus credit (terms). It never affects our data.