Builds / RTX 4090
RTX 4090: Inkling 2 Mini at 59.7 tok/s decode and 1,162 tok/s prefill (8K prompt) on the Xolotl engine, 14 GB XQ quant vs 57 tok/s for the exact 22.9 GB conversion
Last updated:
- GPU(s)
- RTX 4090
Amazon · eBay (used) · Newegg - VRAM / unified memory (GB)
- 24
- CPU
- not stated
Amazon · eBay (used) · Newegg - Motherboard
- not stated
Amazon · eBay (used) · Newegg - RAM
- not stated
Amazon · eBay (used) · Newegg - PSU
- not stated
- Other
- Xolotl (Indra's Mirror engine on Strata; Inkling forward graph ported from llama.cpp PR #32694), measured through Xolotl's server: 8K-token prompt, then 1,024 generated tokens; single stream
- Reported total cost
- not stated
- Power
- not stated
Measured speed (generation tokens/sec)
| Model | Quant | Engine | tok/s | Prompt processing (tok/s) | Details |
|---|---|---|---|---|---|
| Inkling 2 Mini | XQ mixed (~4.0 bpw routed experts, 13 of 24 layers keep official NVFP4; Q8_0 backbone; 13.93 GB), abliterated | Xolotl (xolotl-inkling) | 59.7 | 1,162 @ 8K-token prompt est. TTFT @8k ≈ 7.0 s (calculated from measured prefill) | context: 8K prompt, 1,024 generated📏 Quant publisher's own server-path run. This file is an abliterated (refusal-removed) variant: held-out KL 0.033 / top-1 94.7% vs the original model |
| Inkling 2 Mini | Official precision, exact conversion (NVFP4 + MXFP8 experts, BF16 rest; 22.9 GB) | Xolotl (xolotl-inkling) | 57 | 1,131 @ 8K-token prompt est. TTFT @8k ≈ 7.2 s (calculated from measured prefill) | context: 8K prompt, 1,024 generated📏 Same card and settings, the unmodified parent checkpoint |
Prompt processing = prefill speed (how fast the prompt is read) as reported by the source, with the prompt length it was measured at; blank = the source does not report it. est. TTFT @8k is calculated from the measured prefill speed (8,192 ÷ prefill tok/s), shown only when prefill was measured on a prompt of 512+ tokens. It is not a measurement and ignores load time and caching. measured TTFT is the source's own time-to-first-token. Why prefill matters for coding assistants.
Numbers from the quant publisher's Hugging Face card (21 Nov 2026), run on its own engine rather than llama.cpp. They are about twice what day-one llama.cpp builds report for Inkling 2 Mini on 24-32 GB cards. The publisher says decode is limited by the number of small kernels per token (48 layer passes), not memory bandwidth, so it should improve. DSpark speculative decoding landed in Xolotl the same day but is not in these figures.
- 1× RTX 4090: ≈€3,955 each (≈$4,433), new, approx. price as of 2026-10-16, lowest listing on billiger.de (ASUS TUF Gaming RTX 4090 OG OC 24 GB)
New retail prices in Germany incl. VAT; used cards are often much cheaper. Prices change daily.
Blank fields = not stated in the source. We never estimate build data; GPU street prices above are separately sourced and dated.
Quick answers: Can RTX 4090 run Inkling 2 Mini?