Best home rig for Mistral Medium 3.5 128B
Will it fit my GPU? Best value GPU per euro →
Real, measured generation speeds for Mistral Medium 3.5 128B on home hardware, fastest first. Different quants/engines aren't directly comparable.
| Build | Quant | Engine | tok/s | Cost | |
|---|---|---|---|---|---|
| 4x Mac Studio M5 Ultra cluster (Thunderbolt 5 RDMA): Muse Spark Open at 36.7 tok/s | FP8, 8,192-token context | MLX, 4-node tensor parallel | 23.37 | source | |
| 2x Mac Studio M5 Ultra cluster (Thunderbolt 5 RDMA): Muse Spark Open 816B | FP8, 8,192-token context | MLX, 2-node tensor parallel | 13.69 | source | |
| Mac Studio M5 Ultra 512GB: Qwen3.8 27B and Mistral Medium 3.5 (Apple MLX benchmark) | FP8, 8,192-token context | MLX (mlx_lm.benchmark) | 7.7 | source | |
| Mac Studio M3 Ultra 512GB: Qwen3.8 27B and Mistral Medium 3.5 (Apple MLX benchmark) | FP8, 8,192-token context | MLX (mlx_lm.benchmark) | 4.95 | source |
Your brand here. One sponsor slot on this page, shown to people choosing AI hardware. Sponsor RigBase →
FAQ
How much VRAM does Mistral Medium 3.5 128B need?
The weights are about 78.3 GB at Q4_K_M (128B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is Apple M5 Ultra (512 GB) at 7.7 tok/s.
What is the fastest measured hardware for Mistral Medium 3.5 128B?
4x Apple M5 Ultra: 23.37 tok/s generation (MLX, 4-node tensor parallel, FP8, 8,192-token context), from https://machinelearning.apple.com/research/mlx-distributed-inference.
Which GPUs have measured results for Mistral Medium 3.5 128B?
2x Apple M5 Ultra, 4x Apple M5 Ultra, Apple M3 Ultra, Apple M5 Ultra.
Can it run on… 2x Apple M5 Ultra · 4x Apple M5 Ultra · Apple M3 Ultra · Apple M5 Ultra
Related: what GPU do you need for 2026 models · all builds