<!-- Markdown mirror of https://getrigbase.com/models/nemotron-3-5-bolt-e4b/ . Source of truth is the HTML page. Data licence: CC BY 4.0, please cite RigBase (https://getrigbase.com/models/nemotron-3-5-bolt-e4b/). -->
Source: https://getrigbase.com/models/nemotron-3-5-bolt-e4b/
Last updated: 2026-11-16

[Models](https://getrigbase.com/models/) / Nemotron 3.5 Bolt E4B

# Best home rig for Nemotron 3.5 Bolt E4B

Quick answer

**How much VRAM does Nemotron 3.5 Bolt E4B need?**
The weights are about 4.9 GB at Q4_K_M (8B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is Apple M4 (16 GB) at 28.9 tok/s.

**What is the fastest measured hardware for Nemotron 3.5 Bolt E4B?**
Strix Halo (Radeon 8060S): 96.4 tok/s generation (strix-halo-engine-lab, profile quality, EXL3 M4 4.65 bpw (hyukudan) + MTP drafter), from https://huggingface.co/hyukudan/Nemotron-3.5-Bolt-E4B-EXL3.

Last updated: 16 Nov 2026

Legacy model 8B dense 8 measured results fastest plain decode: 73.5 tok/s  [model info ↗](https://mustafa.net/2026/11/12/nemotron-3-5-bolt-run-locally/)

Older model. For current picks see [current models](https://getrigbase.com/models/).

Real, measured generation speeds for Nemotron 3.5 Bolt E4B on home hardware, fastest first. Different quants/engines aren't directly comparable.

| Build | Quant | Engine | tok/s | Prompt processing (tok/s) | Cost | Source |
|---|---|---|---|---|---|---|
| [Strix Halo (Ryzen AI Max+ 395, 128 GB): Nemotron 3.5 Bolt E4B at 55.5 tok/s on llama.cpp, 96.4 on strix-halo-engine-lab EXL3 with MTP drafter](https://getrigbase.com/builds/strix-halo-128gb-nemotron-35-bolt-e4b-exl3-q4km-engine-lab-hyukudan/) | EXL3 M4 4.65 bpw (hyukudan) + MTP drafter | strix-halo-engine-lab, profile quality | **96.4** ⚡ | – |  | [huggingface.co ↗](https://huggingface.co/hyukudan/Nemotron-3.5-Bolt-E4B-EXL3) |
| [Strix Halo (Ryzen AI Max+ 395, 128 GB): Nemotron 3.5 Bolt E4B at 55.5 tok/s on llama.cpp, 96.4 on strix-halo-engine-lab EXL3 with MTP drafter](https://getrigbase.com/builds/strix-halo-128gb-nemotron-35-bolt-e4b-exl3-q4km-engine-lab-hyukudan/) | Q4_K_M (NVIDIA official 4GB GGUF, 6.18 GB) + MTP drafter | strix-halo-engine-lab, profile default | **91.9** ⚡ | – |  | [huggingface.co ↗](https://huggingface.co/hyukudan/Nemotron-3.5-Bolt-E4B-EXL3) |
| [RTX 2060 Super 8 GB + Core i9-10900K: Qwen3.5 9B Q4_K_M at 54.4 tok/s, Gemma 4 12B at 27.7 (llama.cpp)](https://getrigbase.com/builds/rtx-2060-super-8gb-i9-10900k-qwen35-gemma4-mustafa/) | Q4_K_M (NVIDIA official GGUF, 6.18 GB) | llama.cpp b12064 server (CUDA 12.4, Windows build), -ngl 99 | **73.5** | – |  | [mustafa.net ↗](https://mustafa.net/2026/10/13/local-llm-calculator-tested-four-machines/) |
| [Strix Halo (Ryzen AI Max+ 395, 128 GB): Nemotron 3.5 Bolt E4B at 55.5 tok/s on llama.cpp, 96.4 on strix-halo-engine-lab EXL3 with MTP drafter](https://getrigbase.com/builds/strix-halo-128gb-nemotron-35-bolt-e4b-exl3-q4km-engine-lab-hyukudan/) | Q4_K_M (NVIDIA official 4GB GGUF, 6.18 GB) | strix-halo-engine-lab, profile default | **55.8** | 509 @8K · est. TTFT @8k ≈16.1 s* |  | [huggingface.co ↗](https://huggingface.co/hyukudan/Nemotron-3.5-Bolt-E4B-EXL3) |
| [Strix Halo (Ryzen AI Max+ 395, 128 GB): Nemotron 3.5 Bolt E4B at 55.5 tok/s on llama.cpp, 96.4 on strix-halo-engine-lab EXL3 with MTP drafter](https://getrigbase.com/builds/strix-halo-128gb-nemotron-35-bolt-e4b-exl3-q4km-engine-lab-hyukudan/) | Q4_K_M (NVIDIA official 4GB GGUF, 6.18 GB) | llama.cpp 74a4fb00 | **55.5** | 1,881 @8K · est. TTFT @8k ≈4.4 s* |  | [huggingface.co ↗](https://huggingface.co/hyukudan/Nemotron-3.5-Bolt-E4B-EXL3) |
| [Strix Halo (Ryzen AI Max+ 395, 128 GB): Nemotron 3.5 Bolt E4B at 55.5 tok/s on llama.cpp, 96.4 on strix-halo-engine-lab EXL3 with MTP drafter](https://getrigbase.com/builds/strix-halo-128gb-nemotron-35-bolt-e4b-exl3-q4km-engine-lab-hyukudan/) | EXL3 M4 4.65 bpw (hyukudan, 9.71 GB download, 2.70 GB read per token) | strix-halo-engine-lab, profile quality | **50.2** | 746 @8K · est. TTFT @8k ≈11.0 s* |  | [huggingface.co ↗](https://huggingface.co/hyukudan/Nemotron-3.5-Bolt-E4B-EXL3) |
| [Mac mini M4 (16 GB): Qwen3.5 9B Q4_K_M at 17.7 tok/s, Gemma 4 12B at 13.1 (llama.cpp Metal)](https://getrigbase.com/builds/mac-mini-m4-16gb-qwen35-gemma4-mustafa/) | Q4_K_M (NVIDIA official GGUF, 6.18 GB) | llama.cpp b11567 server (Metal) | **28.9** | – |  | [mustafa.net ↗](https://mustafa.net/2026/10/13/local-llm-calculator-tested-four-machines/) |
| [Intel NUC8i7BEH (Core i7-8559U, CPU only): Nemotron 3.5 Bolt E4B Q4_K_M at 5.0 tok/s, Qwen3.5 4B 5.63, Qwen3.5 9B 3.07 (llama.cpp)](https://getrigbase.com/builds/intel-nuc8i7beh-cpu-only-nemotron-35-bolt-e4b-qwen35-mustafa/) | Q4_K_M (NVIDIA official GGUF, 6.18 GB) | llama.cpp b11561 server (CPU) | **5.0** | – |  | [mustafa.net ↗](https://mustafa.net/2026/11/12/nemotron-3-5-bolt-run-locally/) |

Prompt processing = prefill speed reported by the source, @ prompt length. * est. TTFT @8k is calculated from measured prefill, not measured. [Why it matters](https://getrigbase.com/can-it-run/#prefill).

## Quality cost of each quant for Nemotron 3.5 Bolt E4B

How far each quantization drifts from the full-precision (BF16) model, as published by the quant maker. Lower KL divergence (KLD) is closer to the original; same-top-token % is how often the quant picks the same next token as BF16. Q4_K_M is usually the sweet spot; below Q4 the drift grows fast, which is why RigBase only lists Q4-or-higher speed results.

| Quant | File size | Mean KLD vs BF16 | Same top token |
|---|---|---|---|
| EXL3 M4 4.65 bpw (hyukudan) | 9.71 GB | **0.0367** | 92.94% |
| Q4_K_M (NVIDIA official 4GB GGUF) | 6.18 GB | **0.141** | 87.0% |

Source: [huggingface.co ↗](https://huggingface.co/hyukudan/Nemotron-3.5-Bolt-E4B-EXL3). hyukudan (EXL3 quant publisher) model card: measured through strix-halo-engine-lab's decode path on a held-out 40-document, 8-language set against the official BF16 logits. The EXL3 download is larger because it keeps the embedding tables in BF16; per token it reads 2.70 GB vs 3.13 GB for Q4_K_M. KLD from different publishers, datasets or llama.cpp builds isn't directly comparable.
Ad · this space is available **Sponsor this spot: reach people building local AI rigs.** One clearly-labeled slot per page, no tracking scripts.  [Sponsor this spot →](https://getrigbase.com/advertise/#inquiry)

**Rent before you buy:** try this model on a cloud GPU for a few dollars first: [Runpod](https://runpod.io?ref=77qcpl2c) (referral link; sign up with Google to get the new-user bonus)

## Memory math: how much VRAM / RAM does Nemotron 3.5 Bolt E4B need?

Nemotron 3.5 Bolt E4B has **8B parameters** ([source](https://mustafa.net/2026/11/12/nemotron-3-5-bolt-run-locally/): NVIDIA: 8B total, 4B effective (post-trained from Gemma 4 E4B); llama.cpp reports 7.52B). Weight size = parameters × bits-per-weight ÷ 8, before context (KV cache) and runtime overhead.

| Format | Bits/weight | Weights | Smallest common memory size that holds the weights alone |
|---|---|---|---|
| Q8_0 | 8.50 | 8.5 GB | 12 GB |
| Q6_K | 6.56 | 6.6 GB | 8 GB |
| Q5_K_M | 5.70 | 5.7 GB | 8 GB |
| Q4_K_M | 4.89 | **4.9 GB** | 8 GB |
| Q3_K_M | 4.00 | 4.0 GB | 8 GB |
| Q2_K | 3.16 | 3.2 GB | 8 GB |
| As measured: EXL3 M4 4.65 bpw (hyukudan, 9.71 GB download, 2.70 GB read p | 4.65 | ≈4.7 GB | 8 GB |
| As measured: EXL3 M4 4.65 bpw (hyukudan) + MTP drafter | 4.65 | ≈4.7 GB | 8 GB |

Bits-per-weight for GGUF quants are llama.cpp's own figures ([llama.cpp quantize README](https://github.com/ggml-org/llama.cpp/blob/master/tools/quantize/README.md)). "As measured" rows apply the source's stated bits-per-weight to the parameter count; real files differ slightly because some layers stay at higher precision. Try other combinations in the [VRAM calculator](https://getrigbase.com/can-it-run/).

## FAQ

### How much VRAM does Nemotron 3.5 Bolt E4B need?

The weights are about 4.9 GB at Q4_K_M (8B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is Apple M4 (16 GB) at 28.9 tok/s.

### What is the fastest measured hardware for Nemotron 3.5 Bolt E4B?

Strix Halo (Radeon 8060S): 96.4 tok/s generation (strix-halo-engine-lab, profile quality, EXL3 M4 4.65 bpw (hyukudan) + MTP drafter), from https://huggingface.co/hyukudan/Nemotron-3.5-Bolt-E4B-EXL3.

Related: [what GPU do you need for 2026 models](https://getrigbase.com/guides/what-gpu-for-qwen3-6-gemma-4-gpt-oss/) · [all builds](https://getrigbase.com/builds/)

Speed with and without MTP / DFlash on the same machines: [measured speed-ups](https://getrigbase.com/guides/speculative-decoding-mtp-dflash-speedup/).
