Models / Hermes 4.3 36B
Best home rig for Hermes 4.3 36B
Quick answer
How much VRAM does Hermes 4.3 36B need?
The weights are about 22.0 GB at Q4_K_M (36B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is RTX Spark N1X (82 GB) at 12.0 tok/s.
What is the fastest measured hardware for Hermes 4.3 36B?
RTX Spark N1X: 12.0 tok/s generation (llama.cpp b12110 (CUDA, Windows on Arm), Q4_K_M (21.8 GB)), from https://dev.to/maruhana/a-120b-model-ran-about-4x-faster-than-a-27b-one-8-local-llms-on-a-surface-laptop-ultra-5cpg.
Last updated:
Legacy model36B dense1 measured resultfastest plain decode: 12.0 tok/s model info ↗
Older model. For current picks see current models.
Real, measured generation speeds for Hermes 4.3 36B on home hardware, fastest first. Different quants/engines aren't directly comparable.
| Build | Quant | Engine | tok/s | Prompt processing (tok/s) | Cost | Source |
|---|---|---|---|---|---|---|
| Surface Laptop Ultra (RTX Spark N1X, 128 GB): 8 models on llama.cpp b12110, Flint 104.8 tok/s, gpt-oss 120B 64.2, Qwen3.8 27B 16.5 (maruhana) | Q4_K_M (21.8 GB) | llama.cpp b12110 (CUDA, Windows on Arm) | 12.0 | 733 @512 · est. TTFT @8k ≈11.2 s* | dev.to ↗ |
Prompt processing = prefill speed reported by the source, @ prompt length. * est. TTFT @8k is calculated from measured prefill, not measured. Why it matters.
Memory math: how much VRAM / RAM does Hermes 4.3 36B need?
Hermes 4.3 36B has 36B parameters (source: Nous Research model card, 36B dense). Weight size = parameters × bits-per-weight ÷ 8, before context (KV cache) and runtime overhead.
| Format | Bits/weight | Weights | Smallest common memory size that holds the weights alone |
|---|---|---|---|
| Q8_0 | 8.50 | 38.3 GB | 48 GB |
| Q6_K | 6.56 | 29.5 GB | 32 GB |
| Q5_K_M | 5.70 | 25.7 GB | 32 GB |
| Q4_K_M | 4.89 | 22.0 GB | 24 GB |
| Q3_K_M | 4.00 | 18.0 GB | 24 GB |
| Q2_K | 3.16 | 14.2 GB | 16 GB |
Bits-per-weight for GGUF quants are llama.cpp's own figures (llama.cpp quantize README). "As measured" rows apply the source's stated bits-per-weight to the parameter count; real files differ slightly because some layers stay at higher precision. Try other combinations in the VRAM calculator.
FAQ
How much VRAM does Hermes 4.3 36B need?
The weights are about 22.0 GB at Q4_K_M (36B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is RTX Spark N1X (82 GB) at 12.0 tok/s.
What is the fastest measured hardware for Hermes 4.3 36B?
RTX Spark N1X: 12.0 tok/s generation (llama.cpp b12110 (CUDA, Windows on Arm), Q4_K_M (21.8 GB)), from https://dev.to/maruhana/a-120b-model-ran-about-4x-faster-than-a-27b-one-8-local-llms-on-a-surface-laptop-ultra-5cpg.
Related: what GPU do you need for 2026 models · all builds