Builds / 2x AMD Radeon AI PRO R9700 32 GB
2x Radeon AI PRO R9700 32 GB (Threadripper PRO 5995WX): Qwen3.6 27B at 23.4 tok/s on llama.cpp Q4_K_M, 15.9 on stock vLLM FP8 TP=2, 62.75 with tuned vLLM + MTP (Puget Systems)
Last updated:
- GPU(s)
- 2x AMD Radeon AI PRO R9700 32 GB
Amazon · eBay (used) · Newegg - VRAM / unified memory (GB)
- 64
- CPU
- AMD Ryzen Threadripper PRO 5995WX (64 cores)
Amazon · eBay (used) · Newegg - Motherboard
- ASUS Pro WS WRX80E-SAGE SE
Amazon · eBay (used) · Newegg - RAM
- 128 GB
Amazon · eBay (used) · Newegg - PSU
- not stated
- Other
- Puget Systems test workstation, bare-metal Rocky Linux 10.1, PCIe 5.0 x16 per card, containerized runtimes (Podman/Docker) with PCIe peer-to-peer
- Reported total cost
- not stated
- Power
- about 310-340 W for both cards at one user (measured from the amdgpu driver)
Measured speed (generation tokens/sec)
| Model | Quant | Engine | tok/s | Prompt processing (tok/s) | Details |
|---|---|---|---|---|---|
| Qwen3.6 27B legacy | Q4_K_M (GGUF) | llama.cpp ROCm, layer split across both cards | 23.4 | not stated | context: 500-token prompts📏 NVIDIA GenAI-Perf streaming, 500 input / 200 output tokens, 50 prompts, 3 warm-up requests discarded, concurrency 1; TTFT 382 ms, ITL 41.0 ms; 55.7 tok/s total at 4 users, 56.0 at 8 |
| Qwen3.6 27B legacy | FP8 | vLLM v0.20.2 ROCm (stock image), TP=2 | 15.88 | not stated | context: 500-token prompts📏 NVIDIA GenAI-Perf streaming, 500 input / 200 output tokens, 50 prompts, 3 warm-up requests discarded, concurrency 1; TTFT 337 ms, ITL 61.6 ms; 69.3 tok/s total at 4 users, 156.2 at 8 |
| Qwen3.6 27B legacy | FP8 | vLLM tuned build (vllm-radiance), TP=2 | 32.0 | not stated | context: 500-token prompts📏 NVIDIA GenAI-Perf streaming, 500 input / 200 output tokens, 50 prompts, 3 warm-up requests discarded, concurrency 1; TTFT 136 ms, ITL 30.7 ms; 110.6 tok/s total at 4 users, 198.8 at 8 |
| Qwen3.6 27B legacy | FP8 | vLLM tuned build (vllm-radiance), TP=2 + MTP | 62.75 | not stated | ⚡ spec-decode (MTP)context: 500-token prompts📏 NVIDIA GenAI-Perf streaming, 500 input / 200 output tokens, 50 prompts, 3 warm-up requests discarded, concurrency 1; TTFT 184 ms, ITL 15.1 ms; 186.3 tok/s total at 4 users, 320.2 at 8; Puget reports bit-identical output to plain decode |
Vendor-lab test by Puget Systems (posted 18 Aug 2026, last updated 10 Sep 2026 after a bare-metal re-test: an earlier claim that stock vLLM multi-GPU fails on RDNA 4 came from their KVM/VFIO passthrough setup and was withdrawn). Throughput is the GenAI-Perf generation rate for one user. GPU power for both cards: 310 W (stock vLLM), 312 W (tuned), 315 W (tuned + MTP), 339 W (llama.cpp). The article's single-card FP16 results are a separate RigBase build.
- 2× Radeon AI Pro R9700: ≈€1,848 each (≈$2,071), new, approx. price as of 2026-10-24, lowest listing on billiger.de (PowerColor Radeon AI PRO R9700 32 GB GDDR6)
New retail prices in Germany incl. VAT; used cards are often much cheaper. Prices change daily.
Blank fields = not stated in the source. We never estimate build data; GPU street prices above are separately sourced and dated.
Quick answers: Can 2x Radeon AI Pro R9700 run Qwen3.6 27B?
Speculative-decoding rows on this page are not like-for-like with plain decode. See how much MTP and DFlash really speed things up (measured pairs).