More
Submit build

Builds / RTX 5090

RTX 5090: Boole-32B-A2.5B Preview (1.8-bit Flint, 9 GB) at 565.8 tok/s vs Flint INT4 279.4 on llama.cpp (Precision Numerics)

Last updated:

Source: huggingface.co ↗📏 How measured: reported by the original source; RigBase re-checked the numbers at that source
GPU(s)
RTX 5090
Amazon · eBay (used) · Newegg
VRAM / unified memory (GB)
32
CPU
not stated
Motherboard
not stated
RAM
not stated
PSU
not stated
Other
llama-bench tg128 / pp512, -fa on -ub 1024, GGML_CUDA_GRAPH_OPT=1; Boole needs the Precision Numerics llama.cpp fork (stock llama.cpp cannot load it yet), the baselines ran on upstream llama.cpp
Reported total cost
not stated
Power
not stated

Measured speed (generation tokens/sec)

ModelQuantEnginetok/sPrompt processing (tok/s)Details
Boole 32B-A2.5B PreviewQ2_0 GGUF, 1.8 bits/weight average for routed experts (9.0 GB)llama.cpp fork (PrecisionNumerics)565.818,297 @ 512-token prompt
est. TTFT @8k ≈ 0.45 s (calculated from measured prefill)
context: llama-bench tg128📏 Model publisher's own llama-bench run: tg128 565.8 ± 0.4; time to first token 0.44 s on an 8K prompt; peak 10,057 tok/s prompt+output at 32 concurrent requests (8K in / 1K out)
Flint 40B-A2.5BINT4 (26.0 GB)llama.cpp (upstream)279.412,013 @ 512-token prompt
est. TTFT @8k ≈ 0.68 s (calculated from measured prefill)
context: llama-bench tg128📏 Baseline in the Boole card: tg128 279.4 ± 0.3; TTFT 0.62 s on an 8K prompt; peak 5,755 tok/s at 16 concurrent
Flint 40B-A2.5BUnsloth UD-IQ1_M (11.4 GB)llama.cpp (upstream)264.911,882 @ 512-token prompt
est. TTFT @8k ≈ 0.69 s (calculated from measured prefill)
context: llama-bench tg128📏 Baseline in the Boole card: tg128 264.9 ± 0.2; TTFT 0.61 s on an 8K prompt; peak 7,216 tok/s at 32 concurrent

Prompt processing = prefill speed (how fast the prompt is read) as reported by the source, with the prompt length it was measured at; blank = the source does not report it. est. TTFT @8k is calculated from the measured prefill speed (8,192 ÷ prefill tok/s), shown only when prefill was measured on a prompt of 512+ tokens. It is not a measurement and ignores load time and caching. measured TTFT is the source's own time-to-first-token. Why prefill matters for coding assistants.

Vendor numbers: Precision Numerics made Boole and measured all three rows on one RTX 5090 (card created 10 Nov, updated 13 Nov 2026). Boole is Flint with 21% of routed experts pruned and the rest at 1, 2 or 4 bits. Quality cost on the same card (temperature 1.0, 64K ctx): LiveCodeBench v6 67.3 for Boole vs 77.2 UD-IQ1_M, 81.3 Flint INT4, 81.9 BF16; GSM8K 93.2 vs 94.9 / 95.8 / 96.1. Faster, but clearly lossier.

Current GPU street price (estimate, not the builder's cost)Estimated GPU cost for this build today: ≈€7,140
New retail prices in Germany incl. VAT; used cards are often much cheaper. Prices change daily.
Source: https://huggingface.co/precision-numerics/Boole-32B-A2.5B-Preview-GGUF

Blank fields = not stated in the source. We never estimate build data; GPU street prices above are separately sourced and dated.

Embed this result

Copy into a README or blog. Links back to this build. See badge docs (CC BY 4.0 attribution). The SVG does not invent speeds — numbers stay on the build page.

Markdown

[![Measured on RigBase](https://getrigbase.com/badge/rtx-5090-boole-32b-a2-5b-preview-flint-llamacpp-precision-numerics.svg)](https://getrigbase.com/builds/rtx-5090-boole-32b-a2-5b-preview-flint-llamacpp-precision-numerics/)

HTML

<a href="https://getrigbase.com/builds/rtx-5090-boole-32b-a2-5b-preview-flint-llamacpp-precision-numerics/" rel="noopener"><img src="https://getrigbase.com/badge/rtx-5090-boole-32b-a2-5b-preview-flint-llamacpp-precision-numerics.svg" alt="Measured on RigBase" height="20"/></a>

Preview: Measured on RigBase · How to use

Rent before you buy: try this model on a cloud GPU for a few dollars first: Runpod (referral link; sign up with Google to get the new-user bonus)

Quick answers: Can RTX 5090 run Boole 32B-A2.5B Preview? · Can RTX 5090 run Flint 40B-A2.5B?