<!-- Markdown mirror of https://getrigbase.com/guides/multi-gpu-rtx-3090-local-llm/ . Source of truth is the HTML page. Data licence: CC BY 4.0, please cite RigBase (https://getrigbase.com/guides/multi-gpu-rtx-3090-local-llm/). -->
Source: https://getrigbase.com/guides/multi-gpu-rtx-3090-local-llm/
Last updated: 2026-10-28

# Multi-GPU RTX 3090 for local LLMs: measured 2× / 4× / 6× / 8× setups

Quick answer

**Is multi-GPU RTX 3090 worth it for local LLMs?**
Yes when one 24 GB card is too small. A used 3090 is still the cheap path to 24 GB, and 2–8 cards give 48–192 GB for MoE and 70B-class models. Speeds depend on tensor parallel efficiency and the engine (vLLM TP, llama.cpp multi-GPU).

Last updated: 28 Oct 2026

Used RTX 3090s are still the affordable way to stack 24 GB cards. Below is every multi-3090 result in RigBase, with system RAM when the source stated it.

**Quick answer:** 4×3090: Qwen3.6 35B-A3B at **149** tok/s (AWQ-INT4 (TP=2), vLLM) · 6×3090: Qwen3.8 Flash-Next at **45.9** tok/s · 8×3090: DeepSeek V4 Flash at **95.52** tok/s · Total VRAM is what matters for fit (2×=48 GB, 4×=96 GB, 8×=192 GB); system RAM can stay modest when the model is fully on the GPUs — see the RAM guide.

## Ampere multi-GPU (RTX 3090): Tech2wild measured recipes

[Tech2wild (tonyd2wild)](https://github.com/tonyd2wild) / 2Wild Agency publishes open Ampere recipes that RigBase otherwise under-serves. Below, prose/code figures lead — not count-to-N peaks. Full multi-3090 table follows.

- **2×3090:** MiniCPM-V 4.7 at **198.7** tok/s (INT4 + DSpark, vLLM TP2; [Tech2wild](https://github.com/tonyd2wild/MiniCPM-V-4.7-35B-A3B-2x-RTX-3090))
- **2×3090:** Nex N2.5 mini at **187** tok/s (Int4-Int8Mix, vLLM TP2; [Tech2wild](https://github.com/tonyd2wild/Nex-N2.5-mini-Int4-Int8Mix-2x-RTX-3090))
- **2×3090:** Qwen3.8 27B at **91.2** tok/s (W4A16 AutoRound + DFlash2, vLLM TP2; [Tech2wild](https://github.com/tonyd2wild/Qwen3.8-27B-DFLASH2-AutoRound-W4A16-2x3090))
- **2×3090:** Qwen3.8 27B at **69.9** tok/s (AWQ + DSpark, SGLang TP2; [Tech2wild](https://github.com/tonyd2wild/Qwen3.8-27B-SGLang-vs-vLLM-2x3090))
- **2×3090:** Nanbeige4.3 33B at **62.2** tok/s (W4A16 AutoRound, vLLM TP2; [Tech2wild](https://github.com/tonyd2wild/Nanbeige4.3-33B-W4A16-2x-RTX-3090))
- **4×3090:** Qwen3.8 Flash-Next at **109.5** tok/s (W4A16 + draft, vLLM TP4; [Tech2wild](https://github.com/tonyd2wild/Qwen38-Flash-Next-4x3090))

6 Tony 3090 builds in RigBase · [all tonyd2wild repos](https://github.com/tonyd2wild) · [RTX 3090 hub](https://getrigbase.com/gpus/rtx-3090/)

## All multi-3090 measured results

| GPUs | Model | Build | System RAM | Quant | Engine | tok/s |
|---|---|---|---|---|---|---|
| **2×** | [MiniCPM-V 4.7](https://getrigbase.com/models/minicpm-v-4-7/) | [2x RTX 3090: MiniCPM-V 4.7 INT4 on vLLM TP2, 198.7](https://getrigbase.com/builds/tony-2x-rtx-3090-minicpm-v-4-7-35b-a3b-2x-rtx-3090/) | — | INT4 + DSpark | vLLM TP2, 1 stream | **198.7** ⚡ |
| **2×** | [Nex N2.5 mini](https://getrigbase.com/models/nex-n2-5-mini/) | [2x RTX 3090: Nex N2.5 mini Int4-Int8Mix on vLLM TP](https://getrigbase.com/builds/tony-2x-rtx-3090-nex-n2-5-mini-int4-int8mix-2x-rtx-3090/) | — | Int4-Int8Mix | vLLM TP2, 1 stream | **187** |
| **2×** | [Nex N2.5 mini](https://getrigbase.com/models/nex-n2-5-mini/) | [2x RTX 3090: Nex N2.5 mini Int4-Int8Mix on vLLM TP](https://getrigbase.com/builds/tony-2x-rtx-3090-nex-n2-5-mini-int4-int8mix-2x-rtx-3090/) | — | Int4-Int8Mix | vLLM TP2, 1 stream | **186** |
| **2×** | [Qwen3.8 27B](https://getrigbase.com/models/qwen3-8-27b/) | [2x RTX 3090: Qwen3.8 27B W4A16 AutoRound on vLLM T](https://getrigbase.com/builds/tony-2x-rtx-3090-qwen3-8-27b-dflash2-autoround-w4a16-2x3090/) | — | W4A16 AutoRound + DFlash2 | vLLM TP2, 1 stream | **91.2** ⚡ |
| **2×** | [Qwen3.8 27B](https://getrigbase.com/models/qwen3-8-27b/) | [2x RTX 3090: Qwen3.8 27B AWQ on SGLang TP2, 69.9 t](https://getrigbase.com/builds/tony-2x-rtx-3090-qwen3-8-27b-sglang-vs-vllm-2x3090/) | — | AWQ + DSpark | SGLang TP2, 1 stream | **69.9** ⚡ |
| **2×** | [Nanbeige4.3 33B](https://getrigbase.com/models/nanbeige4-3-33b/) | [2x RTX 3090: Nanbeige4.3 33B W4A16 AutoRound on vL](https://getrigbase.com/builds/tony-2x-rtx-3090-nanbeige4-3-33b-w4a16-2x-rtx-3090/) | — | W4A16 AutoRound | vLLM TP2, 1 stream | **66.2** |
| **2×** | [Nanbeige4.3 33B](https://getrigbase.com/models/nanbeige4-3-33b/) | [2x RTX 3090: Nanbeige4.3 33B W4A16 AutoRound on vL](https://getrigbase.com/builds/tony-2x-rtx-3090-nanbeige4-3-33b-w4a16-2x-rtx-3090/) | — | W4A16 AutoRound | vLLM TP2, 1 stream | **62.2** |
| **4×** | [Qwen3.6 35B-A3B](https://getrigbase.com/models/qwen3-6-35b-a3b/) | [4x RTX 3090 lab: Qwen3.6 on vLLM vs llama.cpp](https://getrigbase.com/builds/4x3090-qwen36-vllm-tp2/) | — | AWQ-INT4 + MTP-3 (TP=2) | vLLM | **225-231** ⚡ |
| **4×** | [Qwen3.6 35B-A3B](https://getrigbase.com/models/qwen3-6-35b-a3b/) | [4x RTX 3090 lab: Qwen3.6 on vLLM vs llama.cpp](https://getrigbase.com/builds/4x3090-qwen36-vllm-tp2/) | — | AWQ-INT4 (TP=2) | vLLM | **149** |
| **4×** | [Qwen3.8 Flash-Next](https://getrigbase.com/models/qwen3-8-flash-next/) | [4x RTX 3090: Qwen3.8 Flash-Next W4A16 on vLLM TP4,](https://getrigbase.com/builds/tony-4x-rtx-3090-qwen38-flash-next-4x3090/) | — | W4A16 + draft | vLLM TP4, 1 stream | **145.8** ⚡ |
| **4×** | [Qwen3.6 35B-A3B](https://getrigbase.com/models/qwen3-6-35b-a3b/) | [4x RTX 3090 lab: Qwen3.6 on vLLM vs llama.cpp](https://getrigbase.com/builds/4x3090-qwen36-vllm-tp2/) | — | IQ4_XS (1 GPU) | llama.cpp | **115-133** |
| **4×** | [Qwen3.8 Flash-Next](https://getrigbase.com/models/qwen3-8-flash-next/) | [4x RTX 3090: Qwen3.8 Flash-Next W4A16 on vLLM TP4,](https://getrigbase.com/builds/tony-4x-rtx-3090-qwen38-flash-next-4x3090/) | — | W4A16 + draft | vLLM TP4, 1 stream | **109.5** ⚡ |
| **6×** | [Qwen3.8 Flash-Next](https://getrigbase.com/models/qwen3-8-flash-next/) | [6x RTX 3090 on a Ryzen 9 7950X (4 on USB4 eGPU doc](https://getrigbase.com/builds/6x-rtx-3090-7950x-qwen38-flash-next-llamacpp-multigpu/) | 192 GB DDR5 (4x48 GB) | UD-Q4_K_XL (~111.3 GB), q8_0 KV | llama.cpp-multigpu fork 1344 | **45.9** |
| **6×** | [Qwen3.8 Flash-Next](https://getrigbase.com/models/qwen3-8-flash-next/) | [6x RTX 3090 on a Ryzen 9 7950X (4 on USB4 eGPU doc](https://getrigbase.com/builds/6x-rtx-3090-7950x-qwen38-flash-next-llamacpp-multigpu/) | 192 GB DDR5 (4x48 GB) | UD-Q4_K_XL (~111.3 GB), q8_0 KV | llama.cpp upstream df03399b8 | **38.5** |
| **8×** | [DeepSeek V4 Flash](https://getrigbase.com/models/deepseek-v4-flash/) legacy | [8x RTX 3090 serving DeepSeek V4 Flash (LimeChain v](https://getrigbase.com/builds/8x-rtx-3090-deepseek-v4-flash-limechain/) | — | see source (FP8 KV cache) | vLLM LimeChain SM86 overlay, | **95.52** |

⚡ = speculative decoding. Buying used 3090s? Read the [buying guide](https://getrigbase.com/guides/used-rtx-3090-buying-guide/). Single-card 3090 vs 5090: [3090 vs 5090](https://getrigbase.com/compare/rtx-3090-vs-rtx-5090-llm/).

## Practical notes

- **Fit first:** weights ≈ params × bits ÷ 8. Qwen3.6 35B-A3B at Q4 is ~22 GB (fits one 3090); Qwen3.8 Flash-Next at Q4 is ~110 GB (needs 6× or a 96 GB card).
- **Engines:** vLLM tensor parallel (TP=2 on 4×3090) and llama.cpp multi-GPU both appear in the table; MTP/spec-decode can roughly double tok/s on code-like text.
- **Power:** each 3090 can spike hard; many builders power-limit (e.g. ~270 W). Plan PSU and cooling for N cards.
- **PCIe:** x8/x8 is usually enough for LLM decode; the bottleneck is VRAM size and memory bandwidth, not CPU.

More: [all RTX 3090 builds](https://getrigbase.com/gpus/rtx-3090/) · [best LLMs for 96 GB](https://getrigbase.com/best/96gb-vram/) · [DGX Spark multi-node recipes](https://getrigbase.com/dgx-spark/)

## FAQ

### Is multi-GPU RTX 3090 worth it for local LLMs?

Yes when one 24 GB card is too small. A used 3090 is still the cheap path to 24 GB, and 2–8 cards give 48–192 GB for MoE and 70B-class models. Speeds depend on tensor parallel efficiency and the engine (vLLM TP, llama.cpp multi-GPU).

### 4× vs 8× RTX 3090 — which do I need?

Match total VRAM to the model weights + KV cache. 4× (96 GB) covers many 30–70B MoE setups at Q4; 6–8× is for 100B+ MoE that must stay on GPU (e.g. Qwen3.8 Flash-Next ~111 GB Q4 on 6×, DeepSeek V4 Flash on 8×). We do not have the same model measured on both 4× and 8× yet, so do not treat those rows as a head-to-head.

### How much system RAM with many 3090s?

When every layer stays on the GPUs, builders run fine with 32 GB of DDR4/DDR5 ECC. RAM matters when you offload experts to system memory. Details: [How much system RAM for local LLMs?](https://getrigbase.com/guides/how-much-system-ram-for-local-llms/)
Ad · this space is available **Sponsor this spot: reach people building local AI rigs.** One clearly-labeled slot per page, no tracking scripts.  [Sponsor this spot →](https://getrigbase.com/advertise/#inquiry)

**Not enough VRAM? Rent before you buy.** Run a multi-GPU model on a rented 24–96 GB cloud GPU by the hour and see the real speed before spending €1,000+. [Try it on Runpod](https://runpod.io?ref=77qcpl2c)
Referral link: if you sign up with Google and add $10, Runpod gives you and RigBase bonus credit ([terms](https://docs.runpod.io/get-started/referrals)). It never affects our data.