Builds / 2x RTX PRO 6000 Blackwell Max-Q 96 GB

2x RTX PRO 6000 Max-Q (192 GB): GLM-5.3 Flash at 177.7 tok/s single stream (vLLM, MTP)

Name: 2x RTX PRO 6000 Max-Q (192 GB): GLM-5.3 Flash at 177.7 tok/s single stream (vLLM, MTP)
Creator: RigBase
License: https://creativecommons.org/licenses/by/4.0/

Source: docker.local-inference-lab.ai ↗📏 How measured: reported by the original source; RigBase re-checked the numbers at that source

GPU(s): 2x RTX PRO 6000 Blackwell Max-Q 96 GB
Amazon · eBay (used) · Newegg
VRAM / unified memory (GB): 192
CPU: AMD Ryzen Threadripper PRO 9955WX (16 cores)
Amazon · eBay (used) · Newegg
Motherboard: ASUS Pro WS WRX90E-SAGE SE
Amazon · eBay (used) · Newegg
RAM: 503 GiB registered DDR5
Amazon · eBay (used) · Newegg
PSU: not stated
Other: Both GPUs PCIe Gen5 x16, 300 W power limit (ran at the limit); vLLM 0.1.dev21823+g19f2c20ed in the karmic-kraken-beta-20261008.2 container, TP 2, FP8 KV cache
Reported total cost: not stated
Power: not stated

Measured speed (generation tokens/sec)

Model	Quant	Engine	tok/s	Prompt processing (tok/s)	Details
GLM-5.3 Flash	modelopt_mixed (ModelOpt mixed precision), FP8 KV	vLLM (karmic-kraken image), TP 2, MTP 3 speculative tokens	177.7	10,450 @ 32k-token prompt est. TTFT @8k ≈ 0.78 s (calculated from measured prefill)	⚡ spec-decode (MTP)context: single stream, empty context📏 lil-bench 1.4 standard: C1 decode; peak aggregate 522 tok/s at 8 concurrent

Prompt processing = prefill speed (how fast the prompt is read) as reported by the source, with the prompt length it was measured at; blank = the source does not report it. est. TTFT @8k is calculated from the measured prefill speed (8,192 ÷ prefill tok/s), shown only when prefill was measured on a prompt of 512+ tokens. It is not a measurement and ignores load time and caching. measured TTFT is the source's own time-to-first-token. Why prefill matters for coding assistants.

Local Inference Lab benchmark run uploaded by @xbrnsfm, 10 Oct 2026, with full serving command and hardware readout.

Current GPU street price (estimate, not the builder's cost)

2× RTX Pro 6000 Blackwell: ≈€16,885 each (≈$18,925), new, approx. price as of 2026-10-24, lowest listing on billiger.de (PNY Quadro RTX Pro 6000 Blackwell 96 GB GDDR7)

Estimated GPU cost for this build today: ≈€33,770
New retail prices in Germany incl. VAT; used cards are often much cheaper. Prices change daily.

Source: https://docker.local-inference-lab.ai/bench/runs/ngFRHNivtY0p

Blank fields = not stated in the source. We never estimate build data; GPU street prices above are separately sourced and dated.

Rent before you buy: try this model on a cloud GPU for a few dollars first: Runpod (referral link; sign up with Google to get the new-user bonus)

Quick answers: Can 2x RTX Pro 6000 Max-Q (Blackwell, 96 GB, 300 W) run GLM-5.3 Flash?