Builds / RTX 5070 12GB
RTX 5070 12GB + 32 GB DDR4: Qwen3.8-Flash-Next 177B at 11.5 tok/s with expert streaming
Source: github.com ↗More source: github.com ↗📏 How measured: reported by the original source; RigBase re-checked the numbers at that source
- GPU(s)
- RTX 5070 12GB
Amazon · eBay (used) · Newegg - VRAM / unified memory (GB)
- 12
- CPU
- AMD Ryzen 5 5600GT
Amazon · eBay (used) · Newegg - Motherboard
- not stated
- RAM
- 32 GB DDR4-2400
Amazon · eBay (used) · Newegg - PSU
- not stated
- Other
- PCIe Gen3, Windows, custom llama.cpp-based expert streaming with a page-locked hot-expert tier
- Reported total cost
- not stated
- Power
- not stated
Measured speed (generation tokens/sec)
| Model | Quant | Engine | tok/s | Details |
|---|---|---|---|---|
| Qwen3.8 Flash-Next | UD-IQ3_XXS, experts streamed from RAM/SSD | llama.cpp-based expert streaming (custom) | 11.5 | |
| Qwen3.8 Flash-Next | UD-IQ3_XXS, long coding prompt (4,892 tokens out) | llama.cpp-based expert streaming (custom) | 10.15 |
llama.cpp Show and tell discussion, 4 Oct 2026. Benchmark ~11.5 tok/s (up from ~7 on the inherited setup); the author saw 14-15 tok/s in normal conversations. Most of the model lives off the GPU (offload), so speed depends heavily on RAM/SSD I/O.
Source: https://github.com/ggml-org/llama.cpp/discussions/29930
https://github.com/MohammadHaishemKhawaja/hashyy
https://github.com/MohammadHaishemKhawaja/hashyy
Blank fields = not stated in the source. We never estimate build data; GPU street prices above are separately sourced and dated.
Rent before you buy: try this model on a cloud GPU for a few dollars first: Runpod (referral link; sign up with Google to get the new-user bonus)