Multi-GPU RTX 3090 for local LLMs: measured 2× / 4× / 6× / 8× setups

Quick answer

Is multi-GPU RTX 3090 worth it for local LLMs?
Yes when one 24 GB card is too small. A used 3090 is still the cheap path to 24 GB, and 2–8 cards give 48–192 GB for MoE and 70B-class models. Speeds depend on tensor parallel efficiency and the engine (vLLM TP, llama.cpp multi-GPU).

Last updated: 28 Oct 2026

Used RTX 3090s are still the affordable way to stack 24 GB cards. Below is every multi-3090 result in RigBase, with system RAM when the source stated it.

Quick answer: 4×3090: Qwen3.6 35B-A3B at 149 tok/s (AWQ-INT4 (TP=2), vLLM) · 6×3090: Qwen3.8 Flash-Next at 45.9 tok/s · 8×3090: DeepSeek V4 Flash at 95.52 tok/s · Total VRAM is what matters for fit (2×=48 GB, 4×=96 GB, 8×=192 GB); system RAM can stay modest when the model is fully on the GPUs — see the RAM guide.

Ampere multi-GPU (RTX 3090): Tech2wild measured recipes

Tech2wild (tonyd2wild) / 2Wild Agency publishes open Ampere recipes that RigBase otherwise under-serves. Below, prose/code figures lead — not count-to-N peaks. Full multi-3090 table follows.

2×3090: MiniCPM-V 4.7 at 198.7 tok/s (INT4 + DSpark, vLLM TP2; Tech2wild)
2×3090: Nex N2.5 mini at 187 tok/s (Int4-Int8Mix, vLLM TP2; Tech2wild)
2×3090: Qwen3.8 27B at 91.2 tok/s (W4A16 AutoRound + DFlash2, vLLM TP2; Tech2wild)
2×3090: Qwen3.8 27B at 69.9 tok/s (AWQ + DSpark, SGLang TP2; Tech2wild)
2×3090: Nanbeige4.3 33B at 62.2 tok/s (W4A16 AutoRound, vLLM TP2; Tech2wild)
4×3090: Qwen3.8 Flash-Next at 109.5 tok/s (W4A16 + draft, vLLM TP4; Tech2wild)

6 Tony 3090 builds in RigBase · all tonyd2wild repos · RTX 3090 hub

All multi-3090 measured results

GPUs	Model	Build	System RAM	Quant	Engine	tok/s
2×	MiniCPM-V 4.7	2x RTX 3090: MiniCPM-V 4.7 INT4 on vLLM TP2, 198.7	—	INT4 + DSpark	vLLM TP2, 1 stream	198.7 ⚡
2×	Nex N2.5 mini	2x RTX 3090: Nex N2.5 mini Int4-Int8Mix on vLLM TP	—	Int4-Int8Mix	vLLM TP2, 1 stream	187
2×	Nex N2.5 mini	2x RTX 3090: Nex N2.5 mini Int4-Int8Mix on vLLM TP	—	Int4-Int8Mix	vLLM TP2, 1 stream	186
2×	Qwen3.8 27B	2x RTX 3090: Qwen3.8 27B W4A16 AutoRound on vLLM T	—	W4A16 AutoRound + DFlash2	vLLM TP2, 1 stream	91.2 ⚡
2×	Qwen3.8 27B	2x RTX 3090: Qwen3.8 27B AWQ on SGLang TP2, 69.9 t	—	AWQ + DSpark	SGLang TP2, 1 stream	69.9 ⚡
2×	Nanbeige4.3 33B	2x RTX 3090: Nanbeige4.3 33B W4A16 AutoRound on vL	—	W4A16 AutoRound	vLLM TP2, 1 stream	66.2
2×	Nanbeige4.3 33B	2x RTX 3090: Nanbeige4.3 33B W4A16 AutoRound on vL	—	W4A16 AutoRound	vLLM TP2, 1 stream	62.2
4×	Qwen3.6 35B-A3B	4x RTX 3090 lab: Qwen3.6 on vLLM vs llama.cpp	—	AWQ-INT4 + MTP-3 (TP=2)	vLLM	225-231 ⚡
4×	Qwen3.6 35B-A3B	4x RTX 3090 lab: Qwen3.6 on vLLM vs llama.cpp	—	AWQ-INT4 (TP=2)	vLLM	149
4×	Qwen3.8 Flash-Next	4x RTX 3090: Qwen3.8 Flash-Next W4A16 on vLLM TP4,	—	W4A16 + draft	vLLM TP4, 1 stream	145.8 ⚡
4×	Qwen3.6 35B-A3B	4x RTX 3090 lab: Qwen3.6 on vLLM vs llama.cpp	—	IQ4_XS (1 GPU)	llama.cpp	115-133
4×	Qwen3.8 Flash-Next	4x RTX 3090: Qwen3.8 Flash-Next W4A16 on vLLM TP4,	—	W4A16 + draft	vLLM TP4, 1 stream	109.5 ⚡
6×	Qwen3.8 Flash-Next	6x RTX 3090 on a Ryzen 9 7950X (4 on USB4 eGPU doc	192 GB DDR5 (4x48 GB)	UD-Q4_K_XL (~111.3 GB), q8_0 KV	llama.cpp-multigpu fork 1344	45.9
6×	Qwen3.8 Flash-Next	6x RTX 3090 on a Ryzen 9 7950X (4 on USB4 eGPU doc	192 GB DDR5 (4x48 GB)	UD-Q4_K_XL (~111.3 GB), q8_0 KV	llama.cpp upstream df03399b8	38.5
8×	DeepSeek V4 Flash legacy	8x RTX 3090 serving DeepSeek V4 Flash (LimeChain v	—	see source (FP8 KV cache)	vLLM LimeChain SM86 overlay,	95.52

⚡ = speculative decoding. Buying used 3090s? Read the buying guide. Single-card 3090 vs 5090: 3090 vs 5090.

Practical notes

Fit first: weights ≈ params × bits ÷ 8. Qwen3.6 35B-A3B at Q4 is ~22 GB (fits one 3090); Qwen3.8 Flash-Next at Q4 is ~110 GB (needs 6× or a 96 GB card).
Engines: vLLM tensor parallel (TP=2 on 4×3090) and llama.cpp multi-GPU both appear in the table; MTP/spec-decode can roughly double tok/s on code-like text.
Power: each 3090 can spike hard; many builders power-limit (e.g. ~270 W). Plan PSU and cooling for N cards.
PCIe: x8/x8 is usually enough for LLM decode; the bottleneck is VRAM size and memory bandwidth, not CPU.

More: all RTX 3090 builds · best LLMs for 96 GB · DGX Spark multi-node recipes

FAQ

Is multi-GPU RTX 3090 worth it for local LLMs?

Yes when one 24 GB card is too small. A used 3090 is still the cheap path to 24 GB, and 2–8 cards give 48–192 GB for MoE and 70B-class models. Speeds depend on tensor parallel efficiency and the engine (vLLM TP, llama.cpp multi-GPU).

4× vs 8× RTX 3090 — which do I need?

Match total VRAM to the model weights + KV cache. 4× (96 GB) covers many 30–70B MoE setups at Q4; 6–8× is for 100B+ MoE that must stay on GPU (e.g. Qwen3.8 Flash-Next ~111 GB Q4 on 6×, DeepSeek V4 Flash on 8×). We do not have the same model measured on both 4× and 8× yet, so do not treat those rows as a head-to-head.

How much system RAM with many 3090s?

When every layer stays on the GPUs, builders run fine with 32 GB of DDR4/DDR5 ECC. RAM matters when you offload experts to system memory. Details: How much system RAM for local LLMs?

Not enough VRAM? Rent before you buy. Run a multi-GPU model on a rented 24–96 GB cloud GPU by the hour and see the real speed before spending €1,000+. Try it on Runpod
Referral link: if you sign up with Google and add $10, Runpod gives you and RigBase bonus credit (terms). It never affects our data.