How much system RAM do you need for local LLMs?
Quick answer
How much system RAM do you need for local LLMs?
Less than most guides claim — if the model weights and KV cache fit in GPU (or unified) memory. Decode then streams from VRAM/HBM bandwidth, not from DDR. 32 GB of system RAM is enough for a multi-GPU rig that keeps the whole model on the cards. You need more RAM when you offload layers/experts to system memory or run big contexts with CPU involvement.
Last updated:
Most “you need 128 GB of RAM” advice assumes CPU offload. When the model fits in VRAM, system RAM matters much less than GPU memory and bandwidth.
Why VRAM beats system RAM for decode speed
Generation speed is limited by how fast the runtime can read weights for each token. GPU memory bandwidth is hundreds of GB/s to TB/s; DDR5 is far lower. So a model that spills to system RAM usually drops to a fraction of the all-GPU speed. The VRAM calculator separates weight size from KV-cache size so you can see what must fit on the card.
32 GB system RAM is a working multi-GPU setup
An 8× RTX 3090 rig (192 GB VRAM) with only 32 GB DDR5 ECC RDIMM works well for models that stay entirely on the GPUs — the host RAM just needs to hold the OS, the runtime, and staging buffers. That is the opposite of “buy 256 GB first” advice, and it is a differentiator vs cloud-shaped shopping lists.
Example from RigBase: EPYC 7F52 jank rig: 4x3090 + 4090 + 4080 (136 GB VRAM) lists 32 GB RDIMM 3200 with 4x RTX 3090, RTX 4090, RTX 4080 running Llama 3.1 70B at 9 tok/s (Q8_0).
Measured builds that state ~32 GB of system RAM
| GPUs | System RAM | Model | Quant | tok/s |
|---|---|---|---|---|
| Apple M6 (12-core GPU, unified) | 32 GB unified | Qwen3.6 35B-A3B | MLX 4-bit | 63.2 |
| RTX 3090 24GB | 32 GB DDR5 | Qwen3.8 27B | Unsloth UD-Q4_K_M (nextn/MTP head em | 56.37 |
| RX 9070 XT 16 GB + Radeon AI PRO R9700 32 GB | 32 GB DDR5-6000 | Qwen3.8 27B | IQ4_XS-v3, q8_0 KV | 31.49 |
| RTX 5070 12GB | 32 GB DDR4-2400 | Qwen3.8 Flash-Next | UD-IQ3_XXS, experts streamed from RA | 11.5 |
| 4x RTX 3090, RTX 4090, RTX 4080 | 32 GB RDIMM 3200 | Llama 3.1 70B legacy | Q8_0 | 9 |
When you do need lots of RAM
- Expert offload: Qwen3.8 Flash-Next on an RTX 5070 12GB streamed experts from RAM/SSD at 11.5 tok/s (UD-IQ3_XXS) — usable, but not 100+ tok/s. Source build.
- Big MoE on few cards: the 6×3090 Flash-Next build uses 192 GB DDR5 alongside 144 GB VRAM.
- Unified memory boxes: Mac Studio / Strix Halo / DGX Spark share one pool — size that pool for weights + KV, not “VRAM + RAM” separately.
Next: multi-GPU 3090 guide · Is Q4 good enough? · best LLMs by VRAM
FAQ
How much system RAM do you need for local LLMs?
Less than most guides claim — if the model weights and KV cache fit in GPU (or unified) memory. Decode then streams from VRAM/HBM bandwidth, not from DDR. 32 GB of system RAM is enough for a multi-GPU rig that keeps the whole model on the cards. You need more RAM when you offload layers/experts to system memory or run big contexts with CPU involvement.
Can an 8× RTX 3090 run with only 32 GB of RAM?
Yes, when the model is fully resident on the GPUs. Builders run 8×3090 (192 GB VRAM) with 32 GB DDR5 ECC RDIMM successfully for that case. If you stream experts from system RAM (common for 100B+ MoE on fewer cards), plan 128–256 GB instead — see the 6×3090 Flash-Next build with 192 GB DDR5.
When does more RAM help?
Expert/layer offload, very long context with CPU KV, running multiple models, or OS + browser + IDE alongside. Unified-memory boxes (Mac, Strix Halo, DGX Spark) share one pool, so “system RAM” and “VRAM” are the same budget.
Diese Seite auf Deutsch: Wie viel System-RAM für lokale LLMs?