Can RTX 4090 run Nemotron 3.5 Bolt 74B-A9B?

Probably only partly: the weights are larger than its memory, so part of the model likely ran from system RAM. Measured 108 tok/s generation with part of the model offloaded to system RAM (llama.cpp, llama-bench tg128 (quant per result); source).

Every measured result for Nemotron 3.5 Bolt 74B-A9B on RTX 4090

Build	Quant / setup	Engine	tok/s
RTX 4090: 2026 models (llamabench community)	llama-bench tg128 (quant per result)	llama.cpp	108 CPU offload	source

Memory: RTX 4090 has 24 GB. Nemotron 3.5 Bolt 74B-A9B weights are about 45.3 GB at Q4_K_M (74.04B params × 4.89 bits ÷ 8, before context/KV cache). Arithmetic, not a measurement; long contexts need more. Check other quants in the VRAM calculator.

Speeds are decode (generation) tokens/sec as reported by each source; quant, engine and context differ, so small gaps are noise.

All measured speeds for Nemotron 3.5 Bolt 74B-A9B: best home rig for Nemotron 3.5 Bolt 74B-A9B.

Other models measured on RTX 4090: Gemma 4 12B · Gemma 4 26B-A4B · Gemma 4 31B · LFM2.5 24B-A2B · Qwen3.5 9B · Qwen3.6 27B · gpt-oss 20B

Not enough VRAM? Rent before you buy. Run Nemotron 3.5 Bolt 74B-A9B on a bigger GPU on a rented 24–96 GB cloud GPU by the hour and see the real speed before spending €1,000+. Try it on RunPod
Referral link: RigBase may earn RunPod credit if you sign up. It never affects our data.

SPONSOR
Your brand here. One sponsor slot on this page, shown to people choosing AI hardware. Sponsor RigBase →