Builds / AMD Radeon RX 9070 XT 16GB
RX 9070 XT 16 GB (Windows, llama.cpp Vulkan): Inkling 2 Mini IQ3_XXS at 20.8 tok/s, slower than dense Qwen3.8 27B IQ3_XXS at 25.6; Qwen3.6 35B-A3B does 142
Last updated:
- GPU(s)
- AMD Radeon RX 9070 XT 16GB
Amazon · eBay (used) · Newegg - VRAM / unified memory (GB)
- 16
- CPU
- AMD Ryzen 7 5700X3D
Amazon · eBay (used) · Newegg - Motherboard
- not stated
Amazon · eBay (used) · Newegg - RAM
- 64 GB
Amazon · eBay (used) · Newegg - PSU
- not stated
- Other
- Windows 10 22H2, Adrenalin 32.0.31041.1004; official llama.cpp win-vulkan-x64 release b12600 (29b52d398) for Inkling, b12345 for the Qwen rows; whole model on the GPU (-ngl 99 -dev Vulkan0), no shared-memory spill for Inkling
- Reported total cost
- not stated
- Power
- not stated
Measured speed (generation tokens/sec)
| Model | Quant | Engine | tok/s | Prompt processing (tok/s) | Details |
|---|---|---|---|---|---|
| Inkling 2 Mini | IQ3_XXS (InterpreticAI, 10.9 GiB) | llama.cpp b12600 Vulkan (Windows) | 20.8 | 403 @ 2048-token prompt est. TTFT @8k ≈ 20.3 s (calculated from measured prefill) | context: llama-bench tg64📏 llama-bench -p 2048 -n 64 -r 1 over fa on/off and ubatch 512/1024: tg64 20.8-21.8, pp2048 395-404 (all within 2%; FA makes no difference because the relative-position bias path skips FA). Listed: FA on, f16 KV, ub 512. llama-server at 8K depth: pp 339-346, tg ~20. Single run per config |
| Qwen3.8 27B | IQ3_XXS | llama.cpp b12345 Vulkan (Windows) | 25.6 | 788 @ 8K depth-token prompt est. TTFT @8k ≈ 10.4 s (calculated from measured prefill) | context: 8K depth📏 Same card, older build: tg 25.6 / pp 788 at 8K depth, 23.9 / 662 at 48K depth; poster notes a little shared-memory spill, so may understate |
| Qwen3.6 35B-A3B | UD-IQ2_M | llama.cpp b12345 Vulkan (Windows) | 142 | not stated | context: llama-bench tg📏 Comparison figure from the same poster: 142 tok/s generation and ~4,000 tok/s pp4096 on the same card |
Prompt processing = prefill speed (how fast the prompt is read) as reported by the source, with the prompt length it was measured at; blank = the source does not report it. est. TTFT @8k is calculated from the measured prefill speed (8,192 ÷ prefill tok/s), shown only when prefill was measured on a prompt of 512+ tokens. It is not a measurement and ignores load time and caching. measured TTFT is the source's own time-to-first-token. Why prefill matters for coding assistants.
Commenter's own measurements on llama.cpp issue #32707 (21 Nov 2026), the day after Inkling 2 Mini was released and llama.cpp support landed. Inkling 2 Mini runs its middle 12 layers three times per token (~5.4B effective active parameters), and on day-one llama.cpp it is GPU-bound and slower than the dense Qwen3.8 27B at both generation and prompt processing. Expect this to change as kernels improve (the poster points at the unfused relative-position bias). The same issue reports ~30 tok/s on an RTX 5090 with the ggml-org IQ4_NL file.
Blank fields = not stated in the source. We never estimate build data; GPU street prices above are separately sourced and dated.