More
Submit build

Models / Boole 32B-A2.5B Preview

Best home rig for Boole 32B-A2.5B Preview

Quick answer

How much VRAM does Boole 32B-A2.5B Preview need?
The weights are about 19.6 GB at Q4_K_M (32B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is RTX 5090 (32 GB) at 565.8 tok/s.

What is the fastest measured hardware for Boole 32B-A2.5B Preview?
RTX 5090: 565.8 tok/s generation (llama.cpp fork (PrecisionNumerics), Q2_0 GGUF, 1.8 bits/weight average for routed experts (9.0 GB)), from https://huggingface.co/precision-numerics/Boole-32B-A2.5B-Preview-GGUF.

Last updated:

Current model32B total · 2.5B active1 measured resultfastest plain decode: 565.8 tok/s model info ↗

Will it fit my GPU? Best value GPU per euro →

Real, measured generation speeds for Boole 32B-A2.5B Preview on home hardware, fastest first. Different quants/engines aren't directly comparable.

BuildQuantEnginetok/sPrompt processing (tok/s)CostSource
RTX 5090: Boole-32B-A2.5B Preview (1.8-bit Flint, 9 GB) at 565.8 tok/s vs Flint INT4 279.4 on llama.cpp (Precision Numerics)Q2_0 GGUF, 1.8 bits/weight average for routed experts (9.0 GB)llama.cpp fork (PrecisionNumerics)565.818,297 @512 · est. TTFT @8k ≈0.45 s*huggingface.co ↗

Prompt processing = prefill speed reported by the source, @ prompt length. * est. TTFT @8k is calculated from measured prefill, not measured. Why it matters.

Rent before you buy: try this model on a cloud GPU for a few dollars first: Runpod (referral link; sign up with Google to get the new-user bonus)

Memory math: how much VRAM / RAM does Boole 32B-A2.5B Preview need?

Boole 32B-A2.5B Preview has 32B parameters, about 2.5B active per token (source: Precision Numerics model card: Flint compressed to 9.0 GB, 21% of routed experts pruned, rest 1/2/4-bit (1.8 bpw average); 32B total / 2.5B active; Apache 2.0; HF repo created 10 Nov 2026). Weight size = parameters × bits-per-weight ÷ 8, before context (KV cache) and runtime overhead.

FormatBits/weightWeightsSmallest common memory size that holds the weights alone
Q8_08.5034.0 GB48 GB
Q6_K6.5626.3 GB32 GB
Q5_K_M5.7022.8 GB24 GB
Q4_K_M4.8919.6 GB24 GB
Q3_K_M4.0016.0 GB16 GB
Q2_K3.1612.6 GB16 GB

Bits-per-weight for GGUF quants are llama.cpp's own figures (llama.cpp quantize README). "As measured" rows apply the source's stated bits-per-weight to the parameter count; real files differ slightly because some layers stay at higher precision. Try other combinations in the VRAM calculator.

Why it can still run on smaller GPUs: it is a mixture-of-experts model, so only about 2.5B of its 32B parameters are used for each generated token. Runtimes such as llama.cpp (--n-cpu-moe), ExLlamaV3 or Project Maya can keep most expert weights in system RAM and only the shared layers and hot experts in VRAM. Then you need VRAM + system RAM ≥ weights, and speed is limited mainly by RAM bandwidth.

FAQ

How much VRAM does Boole 32B-A2.5B Preview need?

The weights are about 19.6 GB at Q4_K_M (32B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is RTX 5090 (32 GB) at 565.8 tok/s.

What is the fastest measured hardware for Boole 32B-A2.5B Preview?

RTX 5090: 565.8 tok/s generation (llama.cpp fork (PrecisionNumerics), Q2_0 GGUF, 1.8 bits/weight average for routed experts (9.0 GB)), from https://huggingface.co/precision-numerics/Boole-32B-A2.5B-Preview-GGUF.

Which GPUs have measured results for Boole 32B-A2.5B Preview?

RTX 5090.

Can it run on… RTX 5090

Related: what GPU do you need for 2026 models · all builds