<!-- Markdown mirror of https://getrigbase.com/models/boole-32b-a2-5b-preview/ . Source of truth is the HTML page. Data licence: CC BY 4.0, please cite RigBase (https://getrigbase.com/models/boole-32b-a2-5b-preview/). -->
Source: https://getrigbase.com/models/boole-32b-a2-5b-preview/
Last updated: 2026-11-17

[Models](https://getrigbase.com/models/) / Boole 32B-A2.5B Preview

# Best home rig for Boole 32B-A2.5B Preview

Quick answer

**How much VRAM does Boole 32B-A2.5B Preview need?**
The weights are about 19.6 GB at Q4_K_M (32B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is RTX 5090 (32 GB) at 565.8 tok/s.

**What is the fastest measured hardware for Boole 32B-A2.5B Preview?**
RTX 5090: 565.8 tok/s generation (llama.cpp fork (PrecisionNumerics), Q2_0 GGUF, 1.8 bits/weight average for routed experts (9.0 GB)), from https://huggingface.co/precision-numerics/Boole-32B-A2.5B-Preview-GGUF.

Last updated: 17 Nov 2026

Current model 32B total · 2.5B active 1 measured result fastest plain decode: 565.8 tok/s  [model info ↗](https://huggingface.co/precision-numerics/Boole-32B-A2.5B-Preview-GGUF)

[Will it fit my GPU?](https://getrigbase.com/can-it-run/) [Best value GPU per euro →](https://getrigbase.com/guides/best-value-gpu-local-llm-europe/)

Real, measured generation speeds for Boole 32B-A2.5B Preview on home hardware, fastest first. Different quants/engines aren't directly comparable.

| Build | Quant | Engine | tok/s | Prompt processing (tok/s) | Cost | Source |
|---|---|---|---|---|---|---|
| [RTX 5090: Boole-32B-A2.5B Preview (1.8-bit Flint, 9 GB) at 565.8 tok/s vs Flint INT4 279.4 on llama.cpp (Precision Numerics)](https://getrigbase.com/builds/rtx-5090-boole-32b-a2-5b-preview-flint-llamacpp-precision-numerics/) | Q2_0 GGUF, 1.8 bits/weight average for routed experts (9.0 GB) | llama.cpp fork (PrecisionNumerics) | **565.8** | 18,297 @512 · est. TTFT @8k ≈0.45 s* |  | [huggingface.co ↗](https://huggingface.co/precision-numerics/Boole-32B-A2.5B-Preview-GGUF) |

Prompt processing = prefill speed reported by the source, @ prompt length. * est. TTFT @8k is calculated from measured prefill, not measured. [Why it matters](https://getrigbase.com/can-it-run/#prefill).
Ad · this space is available **Sponsor this spot: reach people building local AI rigs.** One clearly-labeled slot per page, no tracking scripts.  [Sponsor this spot →](https://getrigbase.com/advertise/#inquiry)

**Rent before you buy:** try this model on a cloud GPU for a few dollars first: [Runpod](https://runpod.io?ref=77qcpl2c) (referral link; sign up with Google to get the new-user bonus)

## Memory math: how much VRAM / RAM does Boole 32B-A2.5B Preview need?

Boole 32B-A2.5B Preview has **32B parameters**, about **2.5B active** per token ([source](https://huggingface.co/precision-numerics/Boole-32B-A2.5B-Preview-GGUF): Precision Numerics model card: Flint compressed to 9.0 GB, 21% of routed experts pruned, rest 1/2/4-bit (1.8 bpw average); 32B total / 2.5B active; Apache 2.0; HF repo created 10 Nov 2026). Weight size = parameters × bits-per-weight ÷ 8, before context (KV cache) and runtime overhead.

| Format | Bits/weight | Weights | Smallest common memory size that holds the weights alone |
|---|---|---|---|
| Q8_0 | 8.50 | 34.0 GB | 48 GB |
| Q6_K | 6.56 | 26.3 GB | 32 GB |
| Q5_K_M | 5.70 | 22.8 GB | 24 GB |
| Q4_K_M | 4.89 | **19.6 GB** | 24 GB |
| Q3_K_M | 4.00 | 16.0 GB | 16 GB |
| Q2_K | 3.16 | 12.6 GB | 16 GB |

Bits-per-weight for GGUF quants are llama.cpp's own figures ([llama.cpp quantize README](https://github.com/ggml-org/llama.cpp/blob/master/tools/quantize/README.md)). "As measured" rows apply the source's stated bits-per-weight to the parameter count; real files differ slightly because some layers stay at higher precision. Try other combinations in the [VRAM calculator](https://getrigbase.com/can-it-run/).

**Why it can still run on smaller GPUs:** it is a mixture-of-experts model, so only about 2.5B of its 32B parameters are used for each generated token. Runtimes such as llama.cpp (--n-cpu-moe), ExLlamaV3 or Project Maya can keep most expert weights in system RAM and only the shared layers and hot experts in VRAM. Then you need **VRAM + system RAM ≥ weights**, and speed is limited mainly by RAM bandwidth.

## FAQ

### How much VRAM does Boole 32B-A2.5B Preview need?

The weights are about 19.6 GB at Q4_K_M (32B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is RTX 5090 (32 GB) at 565.8 tok/s.

### What is the fastest measured hardware for Boole 32B-A2.5B Preview?

RTX 5090: 565.8 tok/s generation (llama.cpp fork (PrecisionNumerics), Q2_0 GGUF, 1.8 bits/weight average for routed experts (9.0 GB)), from https://huggingface.co/precision-numerics/Boole-32B-A2.5B-Preview-GGUF.

### Which GPUs have measured results for Boole 32B-A2.5B Preview?

RTX 5090.

**Can it run on…** [RTX 5090](https://getrigbase.com/can-i-run/boole-32b-a2-5b-preview-on-rtx-5090/)

Related: [what GPU do you need for 2026 models](https://getrigbase.com/guides/what-gpu-for-qwen3-6-gemma-4-gpt-oss/) · [all builds](https://getrigbase.com/builds/)
