Builds / RTX 5090
RTX 5090: Boole-32B-A2.5B Preview (1.8-bit Flint, 9 GB) at 565.8 tok/s vs Flint INT4 279.4 on llama.cpp (Precision Numerics)
Last updated:
- GPU(s)
- RTX 5090
Amazon · eBay (used) · Newegg - VRAM / unified memory (GB)
- 32
- CPU
- not stated
- Motherboard
- not stated
- RAM
- not stated
- PSU
- not stated
- Other
- llama-bench tg128 / pp512, -fa on -ub 1024, GGML_CUDA_GRAPH_OPT=1; Boole needs the Precision Numerics llama.cpp fork (stock llama.cpp cannot load it yet), the baselines ran on upstream llama.cpp
- Reported total cost
- not stated
- Power
- not stated
Measured speed (generation tokens/sec)
| Model | Quant | Engine | tok/s | Prompt processing (tok/s) | Details |
|---|---|---|---|---|---|
| Boole 32B-A2.5B Preview | Q2_0 GGUF, 1.8 bits/weight average for routed experts (9.0 GB) | llama.cpp fork (PrecisionNumerics) | 565.8 | 18,297 @ 512-token prompt est. TTFT @8k ≈ 0.45 s (calculated from measured prefill) | context: llama-bench tg128📏 Model publisher's own llama-bench run: tg128 565.8 ± 0.4; time to first token 0.44 s on an 8K prompt; peak 10,057 tok/s prompt+output at 32 concurrent requests (8K in / 1K out) |
| Flint 40B-A2.5B | INT4 (26.0 GB) | llama.cpp (upstream) | 279.4 | 12,013 @ 512-token prompt est. TTFT @8k ≈ 0.68 s (calculated from measured prefill) | context: llama-bench tg128📏 Baseline in the Boole card: tg128 279.4 ± 0.3; TTFT 0.62 s on an 8K prompt; peak 5,755 tok/s at 16 concurrent |
| Flint 40B-A2.5B | Unsloth UD-IQ1_M (11.4 GB) | llama.cpp (upstream) | 264.9 | 11,882 @ 512-token prompt est. TTFT @8k ≈ 0.69 s (calculated from measured prefill) | context: llama-bench tg128📏 Baseline in the Boole card: tg128 264.9 ± 0.2; TTFT 0.61 s on an 8K prompt; peak 7,216 tok/s at 32 concurrent |
Prompt processing = prefill speed (how fast the prompt is read) as reported by the source, with the prompt length it was measured at; blank = the source does not report it. est. TTFT @8k is calculated from the measured prefill speed (8,192 ÷ prefill tok/s), shown only when prefill was measured on a prompt of 512+ tokens. It is not a measurement and ignores load time and caching. measured TTFT is the source's own time-to-first-token. Why prefill matters for coding assistants.
Vendor numbers: Precision Numerics made Boole and measured all three rows on one RTX 5090 (card created 10 Nov, updated 13 Nov 2026). Boole is Flint with 21% of routed experts pruned and the rest at 1, 2 or 4 bits. Quality cost on the same card (temperature 1.0, 64K ctx): LiveCodeBench v6 67.3 for Boole vs 77.2 UD-IQ1_M, 81.3 Flint INT4, 81.9 BF16; GSM8K 93.2 vs 94.9 / 95.8 / 96.1. Faster, but clearly lossier.
- 1× RTX 5090: ≈€7,140 each (≈$8,003), new, approx. price as of 2026-10-24, lowest listing on billiger.de (MSI GeForce RTX 5090 (lowest variant))
New retail prices in Germany incl. VAT; used cards are often much cheaper. Prices change daily.
Blank fields = not stated in the source. We never estimate build data; GPU street prices above are separately sourced and dated.
Quick answers: Can RTX 5090 run Boole 32B-A2.5B Preview? · Can RTX 5090 run Flint 40B-A2.5B?