Models / Nemotron 3 Super 120B-A12B
Best home rig for Nemotron 3 Super 120B-A12B
Legacy model120B total · 12B active2 measured resultsfastest plain decode: 16.1 tok/s model info ↗
Real, measured generation speeds for Nemotron 3 Super 120B-A12B on home hardware, fastest first. Different quants/engines aren't directly comparable.
| Build | Quant | Engine | tok/s | Cost | Source |
|---|---|---|---|---|---|
| DGX Spark (GB10, 128 GB): 23 models on vLLM at 128K context, Qwen3.6 35B-A3B up to 102 tok/s | + MTP (k=3) | vLLM 0.31.0, 128K ctx, 1 user | 23.0 ⚡ | ctala.github.io ↗ | |
| DGX Spark (GB10, 128 GB): 23 models on vLLM at 128K context, Qwen3.6 35B-A3B up to 102 tok/s | no speculative decoding | vLLM 0.31.0, 128K ctx, 1 user | 16.1 | ctala.github.io ↗ |
Memory math: how much VRAM / RAM does Nemotron 3 Super 120B-A12B need?
Nemotron 3 Super 120B-A12B has 120B parameters, about 12B active per token (source: Ollama library: 120B, 12B active). Weight size = parameters × bits-per-weight ÷ 8, before context (KV cache) and runtime overhead.
| Format | Bits/weight | Weights | Smallest common memory size that holds the weights alone |
|---|---|---|---|
| Q8_0 | 8.50 | 127.5 GB | 128 GB |
| Q6_K | 6.56 | 98.4 GB | 128 GB |
| Q5_K_M | 5.70 | 85.6 GB | 96 GB |
| Q4_K_M | 4.89 | 73.4 GB | 96 GB |
| Q3_K_M | 4.00 | 59.9 GB | 64 GB |
| Q2_K | 3.16 | 47.4 GB | 48 GB |
Bits-per-weight for GGUF quants are llama.cpp's own figures (llama.cpp quantize README). "As measured" rows apply the source's stated bits-per-weight to the parameter count; real files differ slightly because some layers stay at higher precision. Try other combinations in the VRAM calculator.
Why it can still run on smaller GPUs: it is a mixture-of-experts model, so only about 12B of its 120B parameters are used for each generated token. Runtimes such as llama.cpp (--n-cpu-moe), ExLlamaV3 or Project Maya can keep most expert weights in system RAM and only the shared layers and hot experts in VRAM. Then you need VRAM + system RAM ≥ weights, and speed is limited mainly by RAM bandwidth.
FAQ
How much VRAM does Nemotron 3 Super 120B-A12B need?
The weights are about 73.4 GB at Q4_K_M (120B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is DGX Spark (GB10) (128 GB) at 23.0 tok/s.
What is the fastest measured hardware for Nemotron 3 Super 120B-A12B?
DGX Spark (GB10): 23.0 tok/s generation (vLLM 0.31.0, 128K ctx, 1 user, + MTP (k=3)), from https://ctala.github.io/local-llm-agentic-workflows/benchmarks/.
Which GPUs have measured results for Nemotron 3 Super 120B-A12B?
DGX Spark (GB10).
Can it run on… DGX Spark (GB10)
Related: what GPU do you need for 2026 models · all builds