Models / Nemotron 3.5 Lightning 30B-A3B

Best home rig for Nemotron 3.5 Lightning 30B-A3B

Current model30B total · 3B active2 measured resultsfastest plain decode: 70.3 tok/s model info ↗

Will it fit my GPU? Best value GPU per euro →

Real, measured generation speeds for Nemotron 3.5 Lightning 30B-A3B on home hardware, fastest first. Different quants/engines aren't directly comparable.

Build	Quant	Engine	tok/s	Cost	Source
DGX Spark (GB10, 128 GB): 23 models on vLLM at 128K context, Qwen3.6 35B-A3B up to 102 tok/s	+ DSpark speculative decoding (k=3)	vLLM 0.31.0, 128K ctx, 1 user	94.4 ⚡		ctala.github.io ↗
DGX Spark (GB10, 128 GB): 23 models on vLLM at 128K context, Qwen3.6 35B-A3B up to 102 tok/s	no speculative decoding	vLLM 0.31.0, 128K ctx, 1 user	70.3		ctala.github.io ↗

Rent before you buy: try this model on a cloud GPU for a few dollars first: Runpod (referral link; sign up with Google to get the new-user bonus)

Memory math: how much VRAM / RAM does Nemotron 3.5 Lightning 30B-A3B need?

Nemotron 3.5 Lightning 30B-A3B has 30B parameters, about 3B active per token (source: 30B total / 3B active MoE (hybrid Mamba), per DGX Spark benchmark table). Weight size = parameters × bits-per-weight ÷ 8, before context (KV cache) and runtime overhead.

Format	Bits/weight	Weights	Smallest common memory size that holds the weights alone
Q8_0	8.50	31.9 GB	32 GB
Q6_K	6.56	24.6 GB	32 GB
Q5_K_M	5.70	21.4 GB	24 GB
Q4_K_M	4.89	18.4 GB	24 GB
Q3_K_M	4.00	15.0 GB	16 GB
Q2_K	3.16	11.8 GB	12 GB

Bits-per-weight for GGUF quants are llama.cpp's own figures (llama.cpp quantize README). "As measured" rows apply the source's stated bits-per-weight to the parameter count; real files differ slightly because some layers stay at higher precision. Try other combinations in the VRAM calculator.

Why it can still run on smaller GPUs: it is a mixture-of-experts model, so only about 3B of its 30B parameters are used for each generated token. Runtimes such as llama.cpp (--n-cpu-moe), ExLlamaV3 or Project Maya can keep most expert weights in system RAM and only the shared layers and hot experts in VRAM. Then you need VRAM + system RAM ≥ weights, and speed is limited mainly by RAM bandwidth.

FAQ

How much VRAM does Nemotron 3.5 Lightning 30B-A3B need?

The weights are about 18.4 GB at Q4_K_M (30B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is DGX Spark (GB10) (128 GB) at 94.4 tok/s.

What is the fastest measured hardware for Nemotron 3.5 Lightning 30B-A3B?

DGX Spark (GB10): 94.4 tok/s generation (vLLM 0.31.0, 128K ctx, 1 user, + DSpark speculative decoding (k=3)), from https://ctala.github.io/local-llm-agentic-workflows/benchmarks/.

Which GPUs have measured results for Nemotron 3.5 Lightning 30B-A3B?

DGX Spark (GB10).

Can it run on… DGX Spark (GB10)