Builds / RTX 5090
RTX 5090: Qwen3.8 27B NVFP4 GGUF at 84.8 tok/s vs stock Q4_K_M at 81.2 on llama.cpp CUDA, prefill 5,081 vs 3,947 (LibertAI card)
Last updated:
- GPU(s)
- RTX 5090
Amazon · eBay (used) · Newegg - VRAM / unified memory (GB)
- 32
- CPU
- not stated
Amazon · eBay (used) · Newegg - Motherboard
- not stated
Amazon · eBay (used) · Newegg - RAM
- not stated
Amazon · eBay (used) · Newegg - PSU
- not stated
- Other
- CUDA 13.0, sm_120; llama.cpp build 849798132; llama-bench single stream, 3 repetitions; NVFP4 on the MLP linears only (ModelOpt PTQ, group 16), attention / Gated-DeltaNet / embeddings in Q4_K_M or Q8_0
- Reported total cost
- not stated
- Power
- not stated
Measured speed (generation tokens/sec)
| Model | Quant | Engine | tok/s | Prompt processing (tok/s) | Details |
|---|---|---|---|---|---|
| Qwen3.8 27B | NVFP4-Q4_K_M GGUF (LibertAI; 14.72 GiB) | llama.cpp 849798132 (CUDA 13.0) | 84.8 | 5,081 @ 512-token prompt est. TTFT @8k ≈ 1.6 s (calculated from measured prefill) | context: llama-bench tg64📏 llama-bench, 3 reps; batched (llama-batched-bench 512 in / 128 out) total 398 tok/s at 1 stream, 1,520 at 8 |
| Qwen3.8 27B | NVFP4-Q8_0 GGUF (LibertAI; 18.65 GiB) | llama.cpp 849798132 (CUDA 13.0) | 71.8 | 5,169 @ 512-token prompt est. TTFT @8k ≈ 1.6 s (calculated from measured prefill) | context: llama-bench tg64📏 llama-bench, 3 reps; larger Q8_0 attention tensors cost decode speed |
| Qwen3.8 27B | Q4_K_M (stock, made from the same BF16; 15.40 GiB) | llama.cpp 849798132 (CUDA 13.0) | 81.2 | 3,947 @ 512-token prompt est. TTFT @8k ≈ 2.1 s (calculated from measured prefill) | context: llama-bench tg64📏 llama-bench, 3 reps; batched total 376 tok/s at 1 stream, 1,188 at 8 |
Prompt processing = prefill speed (how fast the prompt is read) as reported by the source, with the prompt length it was measured at; blank = the source does not report it. est. TTFT @8k is calculated from the measured prefill speed (8,192 ÷ prefill tok/s), shown only when prefill was measured on a prompt of 512+ tokens. It is not a measurement and ignores load time and caching. measured TTFT is the source's own time-to-first-token. Why prefill matters for coding assistants.
Quant publisher's own same-build comparison on one RTX 5090 (model card, Aug 2026). Native NVFP4 math needs a Blackwell GPU and llama.cpp built with CUDA 12.8+; on older cards the files load but the speed-up does not apply. The decode gain is small (+4%); the bigger wins are prefill (+29%) and batched serving (+28% at 8 parallel requests). tg64 is a short test. Quality: calibrated on English news text only, not tested on code or other languages.
- 1× RTX 5090: ≈€7,140 each (≈$8,003), new, approx. price as of 2026-10-24, lowest listing on billiger.de (MSI GeForce RTX 5090 (lowest variant))
New retail prices in Germany incl. VAT; used cards are often much cheaper. Prices change daily.
Blank fields = not stated in the source. We never estimate build data; GPU street prices above are separately sourced and dated.
Quick answers: Can RTX 5090 run Qwen3.8 27B?