Models / gpt-oss 120B / VRAM
How much VRAM do you need for gpt-oss 120B?
Last updated:
Weight math from the model card, plus every measured RigBase result for gpt-oss 120B. Leave extra room for context (KV cache).
Weight size by quant
| Quant | Bits/weight | Weights only |
|---|---|---|
| Native release format | – | ≈65 GB |
| Q8_0 | 8.50 | 124.3 GB |
| Q6_K | 6.56 | 96.0 GB |
| Q5_K_M | 5.70 | 83.4 GB |
| Q4_K_M | 4.89 | 71.6 GB |
| Q3_K_M | 4.00 | 58.4 GB |
| Q2_K | 3.16 | 46.2 GB |
Bits-per-weight for GGUF rows: llama.cpp quantize README. Parameter count: OpenAI model card: 117B total, 5.1B active. We advise staying at Q4 or above for chat quality — see Is Q4 good enough?.
Measured speeds for gpt-oss 120B
| Hardware | Quant | Engine | tok/s |
|---|---|---|---|
| RTX Pro 6000 (Blackwell) · 96 GB | MXFP4 | llama.cpp | 196.31 |
| Apple M2 Ultra 76-core GPU · 192 GB | MXFP4 | llama.cpp | 79.68 |
| AMD Ryzen AI Max+ PRO 495 (Radeon 8065S iGPU, 40 CUs) · 192 GB | MXFP4 | llama.cpp Vulkan | 55.8 |
| AMD Ryzen AI Max+ PRO 495 (Radeon 8065S iGPU, 40 CUs) · 192 GB | MXFP4, 32K context | llama.cpp Vulkan | 45.1 |
| AMD Ryzen AI Max+ PRO 395 (Radeon 8060S iGPU, 32 GB dedicate · 128 GB | MXFP4 | llama.cpp b11213 Vulkan | 44.7 |
| Nvidia GB10 (unified) · 128 GB | vLLM 0.31.0, 128K ctx, 1 user | 44.4 |
⚡ = speculative decoding (MTP, DFlash…). Full list: best home rig for gpt-oss 120B.
Related
All gpt-oss 120B builds VRAM + KV calculator By VRAM size GGUF vs AWQ vs FP8 vs NVFP4 How much system RAM?
FAQ
How much VRAM do you need for gpt-oss 120B?
Weights alone are about 65 GB in the native release format. Add memory for the KV cache (context) and the runtime. The smallest-memory measured result in RigBase is 96 GB (RTX Pro 6000 (Blackwell)) at 196.31 tok/s. Going below Q4 usually hurts quality for chat; see the Q4 guide.
What is the fastest measured speed for gpt-oss 120B?
RTX Pro 6000 (Blackwell): 196.31 tok/s plain decode (MXFP4, llama.cpp).
Does system RAM replace VRAM?
Only for overflow. When the whole model fits in GPU or unified memory, system RAM barely matters for decode speed. See the system-RAM guide.
Referral link: if you sign up with Google and add $10, Runpod gives you and RigBase bonus credit (terms). It never affects our data.