Builds / RTX 3090
RTX 3090 (Ryzen 7 5700X, llama.cpp v0.7.0): Nemotron 3.5 Bolt 24GB at 89.8 tok/s with 128K context loaded, 82.4 with 102K tokens in the cache; Qwen3.6 35B-A3B Q4_K_M at 162
Last updated:
- GPU(s)
- RTX 3090
Amazon · eBay (used) · Newegg - VRAM / unified memory (GB)
- 24
- CPU
- AMD Ryzen 7 5700X
Amazon · eBay (used) · Newegg - Motherboard
- not stated
- RAM
- 64 GB DDR4-3000
Amazon · eBay (used) · Newegg - PSU
- not stated
- Other
- Stock RTX 3090 24 GB at PCIe 4.0 x16 (no gpu-unlock), 420 W card limit; llama.cpp v0.7.0 (commit 620d222) vs v0.4.0 (5266f24); text only, no mmproj, no drafter; --fit off so the requested context is what loads
- Reported total cost
- not stated
- Power
- 380-407 W mean board power on the Bolt speed runs (420 W limit)
Measured speed (generation tokens/sec)
| Model | Quant | Engine | tok/s | Prompt processing (tok/s) | Details |
|---|---|---|---|---|---|
| Nemotron 3.5 Bolt 74B-A9B | NVIDIA 24GB GGUF (experts IQ2_XXS, 2.0625 bpw; 22.02 GB file), f16 KV | llama.cpp v0.7.0 llama-server (CUDA) | 89.8 | 1,605 @ 4,096-token prompt est. TTFT @8k ≈ 5.1 s (calculated from measured prefill) | context: 131,072 context allocated, 4,096-token prompt📏 llama-server after a 4,096-token prompt, one slot (-np 1), -ngl 99 -fa on -ctk f16 -ctv f16 --fit off; 89.80 tok/s [89.75..89.86] at 128K and identically 89.80 at 32K; peak VRAM 22,266 MiB (2,310 MiB headroom). q8_0 KV: 88.20 at 128K |
| Nemotron 3.5 Bolt 74B-A9B | NVIDIA 24GB GGUF (experts IQ2_XXS, 2.0625 bpw; 22.02 GB file), f16 KV | llama.cpp v0.7.0 llama-bench (CUDA) | 82.42 | not stated | context: llama-bench tg128 at 102,400 tokens depth📏 llama-bench tg128 with 102,400 tokens already in the cache (one repetition, as each cell starts with a 102,400-token prefill); 0.905 of the 91.07 tok/s at 4,096 depth. With q8_0 KV the same cell dropped to 60.61 tok/s, so the source recommends f16 KV on this model |
| Qwen3.6 35B-A3B | Q4_K_M | llama.cpp v0.7.0 llama-bench (CUDA), fully on GPU | 162.24 | 4,362 @ 512-token prompt est. TTFT @8k ≈ 1.9 s (calculated from measured prefill) | context: llama-bench, depth 0📏 same file run on v0.4.0 and v0.7.0 alternately three times: decode 162.94 (v0.4.0) vs 162.24 (v0.7.0); 160.13 at 4,096 depth; prompt +14-16% on v0.7.0 |
| Qwen3.6 35B-A3B | Q4_K_M, 20 layers of experts in system RAM (-ncmoe 20) | llama.cpp v0.7.0 llama-server + --moe-cache-mib 1866 | 73.59 | 301 (prompt length not stated) | context: nine-prompt set, fresh server each run📏 partial offload test of the new MoE expert cache: 70.07 tok/s without the cache, 73.59 [73.54..73.62] with a 1,866 MiB cache (10% of the 18,662 MiB of experts), +5.0% decode, -5.6% prompt; peak VRAM 14,134 MiB |
Prompt processing = prefill speed (how fast the prompt is read) as reported by the source, with the prompt length it was measured at; blank = the source does not report it. est. TTFT @8k is calculated from the measured prefill speed (8,192 ÷ prefill tok/s), shown only when prefill was measured on a prompt of 512+ tokens. It is not a measurement and ignores load time and caching. measured TTFT is the source's own time-to-first-token. Why prefill matters for coding assistants.
InsiderLLM measured guide (published 3 Nov 2026, summarised in its 9 Nov newsletter); predictions were locked in a commit before the runs and raw logs are published. Bolt's official 24GB release keeps 2-bit experts; RigBase lists it because it is NVIDIA's own release format (like the other Bolt rows), not a third-party sub-Q4 quant. Why it fits: only 8 of Bolt's 88 blocks keep a KV cache, so 128K at f16 costs 1,024 MiB. Quality on the source's own 47-item task tied Qwen3.6 35B-A3B (22/47 each). Distinct from the llm-speed RTX 3090 Bolt row (87.75 tok/s), which has no deep-context figure.
Blank fields = not stated in the source. We never estimate build data; GPU street prices above are separately sourced and dated.
Copy into a README or blog. Links back to this build. See badge docs (CC BY 4.0 attribution). The SVG does not invent speeds — numbers stay on the build page.
Markdown
[](https://getrigbase.com/builds/rtx-3090-ryzen-7-5700x-nemotron-35-bolt-128k-insiderllm/)
HTML
<a href="https://getrigbase.com/builds/rtx-3090-ryzen-7-5700x-nemotron-35-bolt-128k-insiderllm/" rel="noopener"><img src="https://getrigbase.com/badge/rtx-3090-ryzen-7-5700x-nemotron-35-bolt-128k-insiderllm.svg" alt="Measured on RigBase" height="20"/></a>
Preview: 
· How to use
Quick answers: Can RTX 3090 run Nemotron 3.5 Bolt 74B-A9B? · Can RTX 3090 run Qwen3.6 35B-A3B?