Builds / Apple M5 Ultra (80-core GPU, unified)
Mac Studio M5 Ultra 256 GB: Qwen3.5 397B-A17B 4-bit MLX (224 GB) at 61.6 tok/s, 44.1 at 131K context (Bohemian Silicon)
Last updated:
- GPU(s)
- Apple M5 Ultra (80-core GPU, unified)
- VRAM / unified memory (GB)
- 256
- CPU
- Apple M5 Ultra (36-core CPU)
- Motherboard
- Mac Studio
Amazon · eBay (used) · Newegg - RAM
- 256 GB unified
- PSU
- not stated
- Other
- macOS 27.2.1, Python 3.12, MLX 0.32.4, mlx-lm 0.32.2; default GPU memory limit (Metal recommends 240 GB on this Mac)
- Reported total cost
- not stated
- Power
- not stated
Measured speed (generation tokens/sec)
| Model | Quant | Engine | tok/s | Prompt processing (tok/s) | Details |
|---|---|---|---|---|---|
| Qwen3.5 397B-A17B legacy | 4-bit MLX (mlx-community, ~224 GB) | mlx-lm 0.32.2 (MLX 0.32.4) | 61.6 | 1,265 @ 8K-token prompt est. TTFT @8k ≈ 6.5 s (calculated from measured prefill) | context: 512-token prompt, 128 generated📏 Bohemian Silicon run 12 Nov 2026: mlx_lm.benchmark, random-token prompt with fixed seed, warm-up before each measurement; 512-token prompt average of 3 runs (prefill 159, varied 141-195); 8,192-token prompt 59.9 tok/s; 16K 58.3, 32K 55.1, 64K 50.3, 131K 44.1 (prefill 1,080; one run each, 64 generated); peak memory 224.3 GB at 512 tokens, 230.6 GB at 131K |
Prompt processing = prefill speed (how fast the prompt is read) as reported by the source, with the prompt length it was measured at; blank = the source does not report it. est. TTFT @8k is calculated from the measured prefill speed (8,192 ÷ prefill tok/s), shown only when prefill was measured on a prompt of 512+ tokens. It is not a measurement and ignores load time and caching. measured TTFT is the source's own time-to-first-token. Why prefill matters for coding assistants.
Measured by Bohemian Silicon, a Prague company that rents out these Macs (owner-run on their own hardware, published 12 Nov 2026). The 397B model fit with the default macOS settings and swap stayed under 45 MB. Random-token prompts give pure speed figures; real tasks can differ. Their October Qwen3 235B test (46 tok/s) used older software, so it is not listed here.
Blank fields = not stated in the source. We never estimate build data; GPU street prices above are separately sourced and dated.