Builds / RTX 5070 Ti 16GB
RTX 5070 Ti 16 GB: KRAFTON Raon-1 mini preview (28.8B-A4.3B) IQ3_KS at ~220-230 tok/s on an ik_llama.cpp fork, 128K context fits
Last updated:
- GPU(s)
- RTX 5070 Ti 16GB
Amazon · eBay (used) · Newegg - VRAM / unified memory (GB)
- 16
- CPU
- not stated
- Motherboard
- not stated
- RAM
- not stated
- PSU
- not stated
- Other
- CUDA 12.8; desktop running on the same GPU; quant only loads on the Mcniiby/ik_llama.cpp-raon fork (ik_llama.cpp quant types + vocab-trimmed output head)
- Reported total cost
- not stated
- Power
- not stated
Measured speed (generation tokens/sec)
| Model | Quant | Engine | tok/s | Prompt processing (tok/s) | Details |
|---|---|---|---|---|---|
| Raon-1 mini Preview | IQ3_KS experts / IQ4_KS_R16 down (13.85 GB, 3.86 bpw), q8_0 KV | ik_llama.cpp-raon fork (CUDA 12.8) | 220 | 4,900 @ 4K-token prompt est. TTFT @8k ≈ 1.7 s (calculated from measured prefill) | context: llama-bench tg128📏 Quant author's card (HF repo created 28 Oct 2026): llama-bench tg128 with -fa 1 -ctk q8_0 -ctv q8_0 '~220-230 tok/s' (lower end listed); a plain port of the model ran ~97 |
| Raon-1 mini Preview | IQ3_KS experts / IQ4_KS_R16 down (13.85 GB, 3.86 bpw), q8_0 KV | ik_llama.cpp-raon fork (CUDA 12.8) | 190 | not stated | context: decode at 16K context📏 Same card: '~190 tok/s' decode at 16K; 128K context fits in 16 GB with q8_0 KV and -ub 256 (~1.2 GB free) |
Prompt processing = prefill speed (how fast the prompt is read) as reported by the source, with the prompt length it was measured at; blank = the source does not report it. est. TTFT @8k is calculated from the measured prefill speed (8,192 ÷ prefill tok/s), shown only when prefill was measured on a prompt of 512+ tokens. It is not a measurement and ignores load time and caching. measured TTFT is the source's own time-to-first-token. Why prefill matters for coding assistants.
Numbers from the quant and fork author (approximate values as published). Quality vs KRAFTON's Q4_K_M (19.7 GB) on 40 x 512 tokens: mean KLD 0.0931, same top token 86.97%. The fork's fastest MoE kernel needs an sm_90+ GPU (RTX 50-series); RTX 30/40 compile but were not tested. Raon-1 mini is a hybrid model (18 linear-attention + 7 MLA layers), so the KV cache stays small at long context.
- 1× RTX 5070 Ti: ≈€1,245 each (≈$1,395), new, approx. price as of 2026-10-24, lowest listing on billiger.de (PNY GeForce RTX 5070 Ti 16 GB (listed as "GDDR6"))
New retail prices in Germany incl. VAT; used cards are often much cheaper. Prices change daily.
https://github.com/Mcniiby/ik_llama.cpp-raon
Blank fields = not stated in the source. We never estimate build data; GPU street prices above are separately sourced and dated.
Quick answers: Can RTX 5070 Ti run Raon-1 mini Preview?