Models / gpt-oss 20B / VRAM
How much VRAM do you need for gpt-oss 20B?
Last updated:
Weight math from the model card, plus every measured RigBase result for gpt-oss 20B. Leave extra room for context (KV cache).
Weight size by quant
| Quant | Bits/weight | Weights only |
|---|---|---|
| Native release format | – | ≈13 GB |
| Q8_0 | 8.50 | 22.2 GB |
| Q6_K | 6.56 | 17.2 GB |
| Q5_K_M | 5.70 | 14.9 GB |
| Q4_K_M | 4.89 | 12.8 GB |
| Q3_K_M | 4.00 | 10.4 GB |
| Q2_K | 3.16 | 8.3 GB |
Bits-per-weight for GGUF rows: llama.cpp quantize README. Parameter count: GGUF param count via llamabench.ai (OpenAI model card: 21B total, 3.6B active). We advise staying at Q4 or above for chat quality — see Is Q4 good enough?.
Measured speeds for gpt-oss 20B
| Hardware | Quant | Engine | tok/s |
|---|---|---|---|
| RTX 5090 · 32 GB | reported as Q4_K_M | llama.cpp 0.7.0-dev | 324.2 |
| RTX 5090 · 32 GB | llama-bench tg128 (quant per result) | llama.cpp | 313 |
| RTX Pro 6000 (Blackwell) · 96 GB | MXFP4 | llama.cpp | 286.91 |
| RTX 5090 · 32 GB | MXFP4 | llama.cpp | 282.51 |
| RTX 4090 · 24 GB | llama.cpp | 279.65 | |
| RTX 3090 · 24 GB | reported as Q4_K_M | llama.cpp 0.7.0-dev | 207.2 |
| RTX 5080 · 16 GB | MXFP4 | llama.cpp | 204.85 |
| RTX 4090 · 24 GB | llama-bench tg128 (quant per result) | llama.cpp | 203 |
| RTX 5070 Ti · 16 GB | MXFP4 | llama.cpp | 189.45 |
| AMD Radeon AI Pro R9700 · 32 GB | llama.cpp | 163.26 | |
| RTX 4070 Ti · 12 GB | llama.cpp | 163.14 | |
| RTX 3090 · 24 GB | MXFP4 | llama.cpp | 161.77 |
⚡ = speculative decoding (MTP, DFlash…). Full list: best home rig for gpt-oss 20B.
Related
All gpt-oss 20B builds VRAM + KV calculator By VRAM size GGUF vs AWQ vs FP8 vs NVFP4 How much system RAM?
FAQ
How much VRAM do you need for gpt-oss 20B?
Weights alone are about 13 GB in the native release format. Add memory for the KV cache (context) and the runtime. The smallest-memory measured result in RigBase is 12 GB (RTX 4070 Ti) at 163.14 tok/s. Going below Q4 usually hurts quality for chat; see the Q4 guide.
What is the fastest measured speed for gpt-oss 20B?
RTX 5090: 324.2 tok/s plain decode (reported as Q4_K_M, llama.cpp 0.7.0-dev).
Does system RAM replace VRAM?
Only for overflow. When the whole model fits in GPU or unified memory, system RAM barely matters for decode speed. See the system-RAM guide.
Referral link: if you sign up with Google and add $10, Runpod gives you and RigBase bonus credit (terms). It never affects our data.