RTX 4090 for local LLMs
24 measured results from 9 builds, each linked to its source.
| Memory | 24 GB per card |
|---|---|
| Memory bandwidth | 1,008 GB/s (memory bandwidth as listed on llamabench) |
| Typical EU price | ≈€3,955 new (ASUS TUF Gaming RTX 4090 OG OC 24 GB, billiger.de, 2026-10-16) |
Can it run? Flint 40B-A2.5B on RTX 4090 GLM 4.7 Flash on RTX 4090 Gemma 4 12B on RTX 4090 Gemma 4 26B-A4B on RTX 4090 Gemma 4 31B on RTX 4090 LFM2.5 24B-A2B on RTX 4090 Muse Glimmer 30B on RTX 4090 Nemotron 3.5 Bolt 74B-A9B on RTX 4090 Qwen3.5 9B on RTX 4090 Qwen3.6 27B on RTX 4090 Qwen3.6 35B-A3B on RTX 4090 gpt-oss 20B on RTX 4090
All builds with the RTX 4090
| Build | GPU(s) | Memory GB (total) | Results |
|---|---|---|---|
| EPYC 7F52 jank rig: 4x3090 + 4090 + 4080 (136 GB VRAM) ⚠ unchecked | 4x RTX 3090, RTX 4090, RTX 4080 | 136 | 1 |
| Single RTX 4090 (llama.cpp CUDA scoreboard) | RTX 4090 24GB | 24 | 1 |
| RTX 4090 desktop (speed comparison vs Macs) | RTX 4090 24GB | 24 | 2 |
| Intel Core Ultra 265K + RTX 4090 | RTX 4090 | 24 | 3 |
| Ryzen 9 7900X + RTX 4090 | RTX 4090 | 24 | 2 |
| RTX 4090: 2026 models (llamabench community) | RTX 4090 | 24 | 8 |
| RTX 4090 (llm-speed signed runs, Oct 2026): Qwen3.6 35B-A3B 161 tok/s, Gemma 4 12B QAT 98 tok/s | RTX 4090 | 24 | 3 |
| RTX 4090 (llm-speed signed run, Oct 2026): Flint 40B-A2.5B at 169-179 tok/s on llama.cpp | RTX 4090 | 24 | 2 |
| RTX 4090: Nemotron 3.5 Bolt (IQ2) and Muse Glimmer 30B | RTX 4090 | 24 | 2 |
All measured results
(MTP, DFlash, DSpark and similar draft-and-verify methods give much higher tok/s on predictable text and are not comparable 1:1 with plain decode)
All RTX 4090 builds: see also all GPUs · best LLMs by memory size · download the dataset.
FAQ
How fast is the RTX 4090 for local LLMs?
The fastest plain-decode result we have is 334.67 tok/s on Llama 3.2 3B (, llama.cpp) in Intel Core Ultra 265K + RTX 4090. Speed depends heavily on the model, quant and engine, so compare like with like in the table. RigBase lists 24 measured results from 9 builds.
How much memory does the RTX 4090 have?
24 GB per card. Memory bandwidth: 1,008 GB/s (memory bandwidth as listed on llamabench).
Which models have been measured on the RTX 4090?
DeepSeek R1 Distill 14B, Flint 40B-A2.5B, GLM 4.7 Flash, Gemma 4 12B, Gemma 4 26B-A4B, Gemma 4 31B, LFM2.5 24B-A2B, Llama 2 7B, Llama 3.1 70B, Llama 3.1 8B, Llama 3.2 3B, Mistral Small 22B, Muse Glimmer 30B, Nemotron 3.5 Bolt 74B-A9B, Qwen3.5 9B, Qwen3.6 27B, Qwen3.6 35B-A3B, gpt-oss 20B.
Where do these numbers come from?
Every row links to a build page with its original public source (benchmark logs, READMEs, forum posts). RigBase does not re-run the benchmarks. Results marked ⚡ use speculative decoding and are not comparable 1:1 with plain decode.
Referral link: if you sign up with Google and add $10, Runpod gives you and RigBase bonus credit (terms). It never affects our data.