Can 4x Apple M5 Ultra run Mistral Medium 3.5 128B?

Q: Can 4x Apple M5 Ultra run Mistral Medium 3.5 128B?

Yes. Measured 23.37 tok/s generation on 4x Apple M5 Ultra, via MLX, 4-node tensor parallel, FP8, 8,192-token context. Source: https://machinelearning.apple.com/research/mlx-distributed-inference

Yes. Measured 23.37 tok/s generation (MLX, 4-node tensor parallel, FP8, 8,192-token context; source).

Every measured result for Mistral Medium 3.5 128B on 4x Apple M5 Ultra

Build	Quant / setup	Engine	tok/s
4x Mac Studio M5 Ultra cluster (Thunderbolt 5 RDMA): Muse Spark Open at 36.7 tok/s	FP8, 8,192-token context	MLX, 4-node tensor parallel	23.37	source

Speeds are decode (generation) tokens/sec as reported by each source; quant, engine and context differ, so small gaps are noise.

All measured speeds for Mistral Medium 3.5 128B: best home rig for Mistral Medium 3.5 128B.

Mistral Medium 3.5 128B on other hardware: 2x Apple M5 Ultra · Apple M3 Ultra · Apple M5 Ultra

Other models measured on 4x Apple M5 Ultra: Muse Spark Open 816B-A42B · Qwen3.8 27B

Not enough VRAM? Rent before you buy. Run Mistral Medium 3.5 128B on a bigger GPU on a rented 24–96 GB cloud GPU by the hour and see the real speed before spending €1,000+. Try it on RunPod
Referral link: RigBase may earn RunPod credit if you sign up. It never affects our data.

SPONSOR
Your brand here. One sponsor slot on this page, shown to people choosing AI hardware. Sponsor RigBase →