What GPU do you need for Qwen3.6, Gemma 4 and gpt-oss locally? (measured numbers, Oct 2026)

Published 19 Oct 2026 · data snapshot from RigBase builds

Every time a new open model comes out, people ask the same thing: will it run on my card, and how fast? Spec-sheet calculators can tell you whether the weights fit. They can't tell you what speed you'll actually get. So here are measured generation speeds (decode tokens/sec) for the models people are running in October 2026, collected from public benchmarks. Every number is linked to the place it came from.

How to read the numbers

• Speeds are generation (decode) tokens/sec at short context unless noted. Prompt processing is a different number and is usually much faster.
• Sources use different engines, quants and llama.cpp builds. Treat gaps under ~10% as noise.
• "llamabench community" rows are aggregated llama-bench tg128 runs from llamabench.ai. Open the source for the exact quant and build.

TL;DR

Details and sources below.

1. gpt-oss 20B: the one model everyone has benchmarked

gpt-oss 20B runs in its native MXFP4 format, so these numbers compare well across cards. These are llama-bench tg128 results from the official llama.cpp gpt-oss guide (discussion #15396), plus Apple results:

Takeaway: a 16 GB card is enough for gpt-oss 20B. Even the 5060 Ti does over 100 tok/s, which is far faster than you can read.

Speed and power: whole-system measurements

Jeff Geerling's ai-benchmarks records peak whole-system power during each run. That data is rare. (His summary table doesn't state the quant.)

The DGX Spark is the slowest box here on raw speed, but it gets the most tokens per watt. The 4090 is the fastest, at about 4× the Spark's peak draw.

2. gpt-oss 120B: you need ~96 GB of fast memory

3. Qwen3.6 35B-A3B: the 24 GB sweet spot

It's a mixture-of-experts model with only ~3B active parameters, so it runs very fast once it fits.

Two things stand out:

• One used RTX 3090 is enough. Giles Thomas got 139.6 tok/s while keeping ~90K tokens of context on the GPU. At the full 262K context, with 12 layers of FFNs moved to the CPU, he still got about 66 tok/s (source).
• A second 3090 helps through vLLM, not llama.cpp. Tensor parallel with AWQ went from 115–133 to 149 tok/s, and adding MTP speculative decoding pushed it to 225–231.

4. Gemma 4 and Qwen3.6 27B: dense models, where bandwidth wins

Dense models read every weight for every token, so speed follows memory bandwidth. Here are llamabench community tg128 results:

Sources: RTX 5090, RTX 4090.

On dense ~30B models, the 5090 is about 1.6× faster than the 4090. On the MoE models the gap is smaller.

For the RTX 3090, llamabench lists 40.2 tok/s on Qwen3.6 27B (source). That row includes some runs with speculative decoding, so don't compare it directly with the table above.

Gemma 4 12B on smaller cards (llamabench community):

So what should you buy?

Based only on the numbers above:

• gpt-oss 20B / Gemma 4 12B: any 16 GB card works. A 12 GB RTX 3060 runs both at usable speeds.
• Qwen3.6 35B-A3B: a single 24 GB card. A used RTX 3090 does 140–153 tok/s.
• Dense ~30B (Gemma 4 31B, Qwen3.6 27B): 24 GB fits them. The RTX 5090 is ~1.6× faster than a 4090 if speed matters.
• gpt-oss 120B: a 96 GB GPU or a big-memory Mac.
• Lowest power: the DGX Spark had the best tokens-per-watt on gpt-oss 20B in Geerling's tests, though it was the slowest box.

Missing your card?

The full dataset has 60+ builds with parts lists, power draw and links to every source: RigBase. You can also check fit on the Can it run? calculator.

If you've run llama-bench on a current model, submit it. It needs a proof link, and that's the only way a number gets in.

Data snapshot: RigBase builds.json, 19 Oct 2026. Corrections welcome. Every row links to its source, so check there first.

More: can X run on Y (measured) · best GPU 2026 · 4090 vs 5090