Models / Raon-1 mini Preview
Best home rig for Raon-1 mini Preview
Quick answer
How much VRAM does Raon-1 mini Preview need?
The weights are about 17.6 GB at Q4_K_M (28.8B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is Radeon RX 7900 XTX (24 GB) at 75.46 tok/s.
What is the fastest measured hardware for Raon-1 mini Preview?
RTX 5070 Ti: 220 tok/s generation (ik_llama.cpp-raon fork (CUDA 12.8), IQ3_KS experts / IQ4_KS_R16 down (13.85 GB, 3.86 bpw), q8_0 KV), from https://huggingface.co/McNiiby/Raon-1-mini-preview-IQ3_KS-GGUF.
Last updated:
Current model28.8B total · 4.3B active5 measured resultsfastest plain decode: 220 tok/s model info ↗
Will it fit my GPU? Best value GPU per euro →
Real, measured generation speeds for Raon-1 mini Preview on home hardware, fastest first. Different quants/engines aren't directly comparable.
Prompt processing = prefill speed reported by the source, @ prompt length. * est. TTFT @8k is calculated from measured prefill, not measured. Why it matters.
Quality cost of each quant for Raon-1 mini Preview
How far each quantization drifts from the full-precision (BF16) model, as published by the quant maker. Lower KL divergence (KLD) is closer to the original; same-top-token % is how often the quant picks the same next token as BF16. Q4_K_M is usually the sweet spot; below Q4 the drift grows fast, which is why RigBase only lists Q4-or-higher speed results.
| Quant | File size | Mean KLD vs BF16 | Same top token |
|---|---|---|---|
| IQ3_KS experts, trimmed head (released) below Q4 | 13.85 GB | 0.0931 | 86.97% |
| IQ3_KS experts, full head below Q4 | 13.96 GB | 0.0923 | 86.97% |
| IQ4_KS experts, full head (not released) | 16.26 GB | 0.0496 | 91.12% |
Source: huggingface.co ↗. McNiiby (ik_llama.cpp quant publisher) card: KL divergence vs KRAFTON's Q4_K_M GGUF (19.7 GB, not BF16) on 40 x 512 tokens of held-out text; PPL ratio 1.040 / 1.040 / 1.001. These quant types load only on ik_llama.cpp forks. KLD from different publishers, datasets or llama.cpp builds isn't directly comparable.
Memory math: how much VRAM / RAM does Raon-1 mini Preview need?
Raon-1 mini Preview has 28.8B parameters, about 4.3B active per token (source: KRAFTON model card: 28.8B total / 4.3B activated MoE (6 of 128 routed experts + 1 shared), hybrid 18 Raon Delta Attention + 7 MLA layers, 262K context, Apache 2.0, trained from scratch; preview release (HF repo created 21 Oct 2026); HF API lists 28.78B params). Weight size = parameters × bits-per-weight ÷ 8, before context (KV cache) and runtime overhead.
| Format | Bits/weight | Weights | Smallest common memory size that holds the weights alone |
|---|---|---|---|
| Q8_0 | 8.50 | 30.6 GB | 32 GB |
| Q6_K | 6.56 | 23.6 GB | 24 GB |
| Q5_K_M | 5.70 | 20.5 GB | 24 GB |
| Q4_K_M | 4.89 | 17.6 GB | 24 GB |
| Q3_K_M | 4.00 | 14.4 GB | 16 GB |
| Q2_K | 3.16 | 11.4 GB | 12 GB |
| As measured: IQ3_KS experts / IQ4_KS_R16 down (13.85 GB, 3.86 bpw), q8_0 | 3.86 | ≈13.9 GB | 16 GB |
| As measured: IQ3_KS experts / IQ4_KS_R16 down (13.85 GB, 3.86 bpw), q8_0 | 3.86 | ≈13.9 GB | 16 GB |
Bits-per-weight for GGUF quants are llama.cpp's own figures (llama.cpp quantize README). "As measured" rows apply the source's stated bits-per-weight to the parameter count; real files differ slightly because some layers stay at higher precision. Try other combinations in the VRAM calculator.
Why it can still run on smaller GPUs: it is a mixture-of-experts model, so only about 4.3B of its 28.8B parameters are used for each generated token. Runtimes such as llama.cpp (--n-cpu-moe), ExLlamaV3 or Project Maya can keep most expert weights in system RAM and only the shared layers and hot experts in VRAM. Then you need VRAM + system RAM ≥ weights, and speed is limited mainly by RAM bandwidth.
Measured on setups with less GPU memory than the Q4_K_M weights (so part of the model lived in system RAM or was streamed from disk):
- RTX 5070 Ti 16GB (16 GB): 220 tok/s (IQ3_KS experts / IQ4_KS_R16 down (13.85 GB, 3.86 bpw), q8_0 KV)
- RTX 5070 Ti 16GB (16 GB): 190 tok/s (IQ3_KS experts / IQ4_KS_R16 down (13.85 GB, 3.86 bpw), q8_0 KV)
- RTX 5060 Ti 16GB (16 GB): 42.1 tok/s (IQ4_XS GGUF (KRAFTON, 14.97 GiB) + DFlash2 Q8_0 drafter (699 MiB))
FAQ
How much VRAM does Raon-1 mini Preview need?
The weights are about 17.6 GB at Q4_K_M (28.8B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is Radeon RX 7900 XTX (24 GB) at 75.46 tok/s.
What is the fastest measured hardware for Raon-1 mini Preview?
RTX 5070 Ti: 220 tok/s generation (ik_llama.cpp-raon fork (CUDA 12.8), IQ3_KS experts / IQ4_KS_R16 down (13.85 GB, 3.86 bpw), q8_0 KV), from https://huggingface.co/McNiiby/Raon-1-mini-preview-IQ3_KS-GGUF.
Which GPUs have measured results for Raon-1 mini Preview?
RTX 5060 Ti 16GB, RTX 5070 Ti, Radeon RX 7900 XTX.
Can it run on… RTX 5060 Ti 16GB · RTX 5070 Ti · Radeon RX 7900 XTX
Related: what GPU do you need for 2026 models · all builds