Best local LLMs for 16 GB of VRAM or unified memory
Measured speeds from builds where the model ran on more than 12 and up to 16 GB of GPU or unified memory. Each number links to its build and source.
8 GB · 12 GB · 16 GB · 24 GB · 32 GB · 48 GB · 64 GB · 96 GB · 128 GB or more
GPUs in this tier: RTX 5080 · RTX 5070 Ti · RTX 5060 Ti 16GB · RTX 4060 Ti 16GB · RTX A4000
Fastest measured result per model
| Model | Plain decode tok/s | With speculative decoding ⚡ | Results |
|---|---|---|---|
| gpt-oss 20B | 204.85 (MXFP4, RTX 5080) | – | 4 |
| Qwen3.8 27B | – | 139 (3-bit GSQ, RTX 5070 Ti) | 2 |
| Gemma 4 12B | 87.6 (llama-bench tg128 (quant per result), RTX 5070 Ti) | – | 1 |
| LFM2.5 24B-A2B | 131-150 (llama-bench tg128 (quant per result), RTX 4060 Ti 16GB) | – | 1 |
| Muse Glimmer 30B | 21.42 (UD-IQ3_XXS, RTX A4000) | – | 1 |
| Llama 3.2 3B legacy | 204.45 (, RTX 4080) | – | 3 |
| Llama 3.3 70B legacy | – | – | 1 |
Excludes results the source marks pending and results where part of the model sat in system RAM.
All results in this tier
(MTP, DFlash, DSpark and similar draft-and-verify methods give much higher tok/s on predictable text and are not comparable 1:1 with plain decode)
| Model | Build | Ran on | Quant | Engine | tok/s | Details |
|---|---|---|---|---|---|---|
| Gemma 4 12B | RTX 5070 Ti: 2026 models (llamabench community) | RTX 5070 Ti (16 GB) | llama-bench tg128 (quant per result) | llama.cpp | 87.6 | |
| LFM2.5 24B-A2B | RTX 4060 Ti 16GB: 2026 models (llamabench community) | RTX 4060 Ti 16GB (16 GB) | llama-bench tg128 (quant per result) | llama.cpp | 131-150 | |
| Muse Glimmer 30B | RTX A4000 16GB (llm-speed signed run, Oct 2026): Muse Glimmer 30B UD-IQ3_XXS at 21.4 tok/s | RTX A4000 (16 GB) | UD-IQ3_XXS | llama.cpp 0.7.0-dev (build 11803) | 21.42 | |
| Qwen3.8 27B | Dual RTX 5070 Ti on Ryzen 9 9900X (capy-flash maintainer rig) | RTX 5070 Ti (16 GB) | 3-bit GSQ | capy-flash (1 GPU), MTP, decode at 8k prompt | 139 | ⚡ spec-decode (MTP)context: 8k prompt📏 Maintainer/user decode tok/s from the capy-flash README |
| Qwen3.8 27B | RTX 4060 Ti 16GB: 2026 models (llamabench community) | RTX 4060 Ti 16GB (16 GB) | llama-bench tg128 (quant per result) | llama.cpp | 10.7-17.6 | partly in system RAM |
| gpt-oss 20B | Single RTX 5080 16GB | RTX 5080 (16 GB) | MXFP4 | llama.cpp | 204.85 | |
| gpt-oss 20B | Single RTX 5070 Ti 16GB | RTX 5070 Ti (16 GB) | MXFP4 | llama.cpp | 189.45 | |
| gpt-oss 20B | Single RTX 5060 Ti 16GB | RTX 5060 Ti 16GB (16 GB) | MXFP4 | llama.cpp | 111.51 | |
| gpt-oss 20B | Intel Core Ultra 265K + RTX A4000 | RTX A4000 (16 GB) | llama.cpp | 93.55 | ||
| Llama 3.2 3B legacy | Ryzen 9 7950X + RTX 4080 | RTX 4080 (16 GB) | llama.cpp | 204.45 | ||
| Llama 3.2 3B legacy | Ryzen 9 7950X + RTX 4070 Ti Super | RTX 4070 Ti Super (16 GB) | llama.cpp | 198.95 | ||
| Llama 3.2 3B legacy | Intel Core Ultra 265K + RTX A4000 | RTX A4000 (16 GB) | llama.cpp | 149.44 | ||
| Llama 3.3 70B legacy | Single RTX 4060 Ti 16GB + 64 GB RAM (partial offload) ⚠ unchecked | RTX 4060 Ti 16GB (16 GB) | Q5_K_M | LM Studio / Kobold | 1.5 | partly in system RAM |
What fits by memory math
Q4_K_M weights (4.8944 bits per weight, per the llama.cpp quantize table) that fit in 16 GB with ~5% headroom, before KV cache: Devstral Small 2 24B ≈15 GB, LFM2.5 24B-A2B ≈15 GB, gpt-oss 20B ≈13 GB, Mellum 2.1 12B-A2.5B ≈7 GB, Gemma 4 12B ≈7 GB, Qwen3.5 9B ≈5 GB, Llama 3.1 8B legacy ≈5 GB.
FAQ
Which local LLMs have been measured on 16 GB of VRAM?
Llama 3.3 70B, gpt-oss 20B, Llama 3.2 3B, Gemma 4 12B, LFM2.5 24B-A2B, Qwen3.8 27B, Muse Glimmer 30B (13 results).
What fits in 16 GB of VRAM?
By memory math at Q4_K_M (4.8944 bits per weight, llama.cpp), weights only, before KV cache and context: Devstral Small 2 24B (≈15 GB), LFM2.5 24B-A2B (≈15 GB), gpt-oss 20B (≈13 GB), Mellum 2.1 12B-A2.5B (≈7 GB), Gemma 4 12B (≈7 GB), Qwen3.5 9B (≈5 GB), Llama 3.1 8B (≈5 GB). Leave a few GB for context.
Are speculative-decoding results comparable?
Not 1:1. MTP, DFlash and similar methods draft tokens that the main model verifies, which can double tok/s on predictable text. This page lists them in a separate column; tick the box to hide them in the full table.
Referral link: if you sign up with Google and add $10, Runpod gives you and RigBase bonus credit (terms). It never affects our data.