More
Submit build

Builds / RTX Pro 6000 (Blackwell)

RTX PRO 6000 Blackwell 96 GB: Nemotron 3.5 Bolt 74B-A9B mixed NVFP4/FP8 on vLLM 0.32 at 121.7 tok/s for one user (150.9 with two cards TP=2), 4.0M-token KV cache

Last updated:

Source: docs.understand.tech ↗📏 How measured: reported by the original source; RigBase re-checked the numbers at that source
GPU(s)
RTX Pro 6000 (Blackwell)
Amazon · eBay (used) · Newegg
VRAM / unified memory (GB)
96
CPU
not stated
Motherboard
not stated
RAM
not stated
PSU
not stated
Other
vLLM 0.32.0, checkpoint primitive-ai/Nemotron-3.5-Bolt-74B-A9B-mixed-NVFP4-FP8 (46.7 GB), 44.0 GiB weights per GPU, KV cache 3,993,951 tokens, gpu-memory-utilization 0.92
Reported total cost
not stated
Power
not stated

Measured speed (generation tokens/sec)

ModelQuantEnginetok/sPrompt processing (tok/s)Details
Nemotron 3.5 Bolt 74B-A9Bmixed NVFP4 + FP8 (primitive-ai)vLLM 0.32.0, TP=1, b12x MoE backend121.7not statedcontext: 262,144 ctx, chat workload, 1 session📏 Understand Tech serving test, one chat session, vLLM 0.32.0 (V2 model runner), b12x MoE backend, FlashInfer attention + Mamba backends, prefix caching on, 262,144 ctx; first token p95 0.6 s; per user 55.1 at 8 sessions, 24.0 at 32; chat throughput tops out near 859 tok/s at 64 sessions; repeats spread under 3%

Hardware test by Understand Tech (AI-in-a-box documentation; page undated, vLLM 0.32.0). Same page, two cards with TP=2: 150.9 tok/s for one user (+24%), KV cache 13.3M tokens, about 856 tok/s chat ceiling; the authors recommend TP=1 per GPU because it serves 67-80% more chat sessions per GPU. The page also reports 107.6 tok/s for 1 request on the model card's own test vs 112.7 on the model card. CPU, RAM and board are not stated.

Source: https://docs.understand.tech/documentation/english-version/ai-in-a-box/hardware-experiments/nemotron-3.5-bolt-74b-a9b

Blank fields = not stated in the source. We never estimate build data; GPU street prices above are separately sourced and dated.

Embed this result

Copy into a README or blog. Links back to this build. See badge docs (CC BY 4.0 attribution). The SVG does not invent speeds — numbers stay on the build page.

Markdown

[![Measured on RigBase](https://getrigbase.com/badge/rtx-pro-6000-blackwell-nemotron-35-bolt-nvfp4-fp8-vllm-understand-tech.svg)](https://getrigbase.com/builds/rtx-pro-6000-blackwell-nemotron-35-bolt-nvfp4-fp8-vllm-understand-tech/)

HTML

<a href="https://getrigbase.com/builds/rtx-pro-6000-blackwell-nemotron-35-bolt-nvfp4-fp8-vllm-understand-tech/" rel="noopener"><img src="https://getrigbase.com/badge/rtx-pro-6000-blackwell-nemotron-35-bolt-nvfp4-fp8-vllm-understand-tech.svg" alt="Measured on RigBase" height="20"/></a>

Preview: Measured on RigBase · How to use

Rent before you buy: try this model on a cloud GPU for a few dollars first: Runpod (referral link; sign up with Google to get the new-user bonus)

Quick answers: Can RTX Pro 6000 (Blackwell, 96 GB) run Nemotron 3.5 Bolt 74B-A9B?