Builds / Nvidia GB10 (unified)
DGX Spark (GB10): Nemotron 3.5 Bolt 74B-A9B 24GB IQ2 on llama.cpp with NVIDIA's LiLiCorr drafter, 80.4 tok/s greedy (38.3 without speculation), 262K context
Last updated:
- GPU(s)
- Nvidia GB10 (unified)
- VRAM / unified memory (GB)
- 128
- CPU
- Nvidia GB10 Grace (Arm)
Amazon · eBay (used) · Newegg - Motherboard
- not stated
- RAM
- 128 GB LPDDR5X unified
- PSU
- not stated
- Other
- 1 GB10 box, CUDA 13.0, driver 580.159.03, llama.cpp 1268eda (llama-server, --parallel 1), NVIDIA's published 128 GB serve command unchanged; about 30 GB GPU memory with 262K ctx + vision projector + drafter
- Reported total cost
- not stated
- Power
- not stated
Measured speed (generation tokens/sec)
| Model | Quant | Engine | tok/s | Prompt processing (tok/s) | Details |
|---|---|---|---|---|---|
| Nemotron 3.5 Bolt 74B-A9B | NVIDIA 24GB GGUF (2-bit IQ2) + LiLiCorr drafter GGUF | llama.cpp 1268eda llama-server + LiLiCorr, 7 draft tokens | 80.4 | 1,129 @ 32,768-token prompt est. TTFT @8k ≈ 7.3 s (calculated from measured prefill) | ⚡ spec-decode (LiLiCorr)context: 262K ctx, single stream, thinking on📏 512 forced output tokens, greedy, mean of 4 tasks (math 108.2, code 81.2, summary 69.6, prose 62.6), pooled over 4 runs; sampled (T=1.0, top_p 0.95) 74.1; mean accepted length 4.64; server timings.predicted_per_second; 72.3 tok/s with a 32K prompt, 70.0 at 131K |
| Nemotron 3.5 Bolt 74B-A9B | NVIDIA 24GB GGUF (2-bit IQ2) | llama.cpp 1268eda llama-server, no speculation | 38.3 | 1,276 @ 32,768-token prompt est. TTFT @8k ≈ 6.4 s (calculated from measured prefill) | context: 262K ctx, single stream, thinking on📏 Same test, one run; 36.4 tok/s with a 32K prompt, 31.1 at 131K |
Prompt processing = prefill speed (how fast the prompt is read) as reported by the source, with the prompt length it was measured at; blank = the source does not report it. est. TTFT @8k is calculated from the measured prefill speed (8,192 ÷ prefill tok/s), shown only when prefill was measured on a prompt of 512+ tokens. It is not a measurement and ignores load time and caching. measured TTFT is the source's own time-to-first-token. Why prefill matters for coding assistants.
Owner-run recipe repo (README committed 11 Nov 2026, results.json and bench scripts published). 6 drafts tie 7 (80.6 vs 80.4); 4 and 5 drafts give 74.8 and 78.1. The drafter costs about 12% prefill. The author notes outputs are not bit-identical to no-speculation runs, and multi-user throughput was not tested. Compare with Mia's TensorFold recipe on the same box (71 tok/s on her card; 67.9 in niklaslenz.com's independent re-run) - different engines and prompts.
Blank fields = not stated in the source. We never estimate build data; GPU street prices above are separately sourced and dated.
Quick answers: Can DGX Spark (GB10) run Nemotron 3.5 Bolt 74B-A9B?
Speculative-decoding rows on this page are not like-for-like with plain decode. See how much MTP and DFlash really speed things up (measured pairs).