Apple M5 Pro for local LLMs
Quick answer
How fast is the Apple M5 Pro for local LLMs?
The fastest plain-decode result we have is 100.24 tok/s on Qwen3.6 35B-A3B (MLX 4-bit (mlx-community/Qwen3.6-35B-A3B-4bit), mlx-lm (MLX 0.32.x)) in Mac mini M5 Pro 64GB (MLX 4-bit): Qwen3.6 35B-A3B at 100.2 tok/s, Gemma 4 26B-A4B at 85.1, Qwen3.8 27B at 17.7 (mean over 1,000 rounds). Speed depends heavily on the model, quant and engine, so compare like with like in the table. RigBase lists 10 measured results from 3 builds.
Last updated:
10 measured results from 3 builds, each linked to its source.
| Memory | unified memory: 24–64 GB in our builds |
|---|---|
| Memory bandwidth | not listed |
| Typical EU price | not tracked yet |
All builds with the Apple M5 Pro
| Build | GPU(s) | Memory GB (total) | Results |
|---|---|---|---|
| Mac mini M5 Pro 64GB (MLX 4-bit): Qwen3.6 35B-A3B at 100.2 tok/s, Gemma 4 26B-A4B at 85.1, Qwen3.8 27B at 17.7 (mean over 1,000 rounds) | Apple M5 Pro (unified, Mac mini) | 64 | 3 |
| Mac mini M5 Pro 64GB (llama.cpp IQ4_NL): Inkling 2 Mini at 29.95 tok/s short context, 17.39 tok/s at 128k (pratikgx) | Apple M5 Pro (unified, Mac mini) | 64 | 5 |
| MacBook Pro M5 Pro 24 GB: Qwen3.8 Flash-Next (125B-A6B) IQ3_XXS at 14.7 tok/s with experts streamed from SSD (MoEShift) | Apple M5 Pro (unified) | 24 | 2 |
All measured results
(MTP, DFlash, DSpark and similar draft-and-verify methods give much higher tok/s on predictable text and are not comparable 1:1 with plain decode)
| Model | Build | Ran on | Quant | Engine | tok/s | Details |
|---|---|---|---|---|---|---|
| Gemma 4 26B-A4B | Mac mini M5 Pro 64GB (MLX 4-bit): Qwen3.6 35B-A3B at 100.2 tok/s, Gemma 4 26B-A4B at 85.1, Qwen3.8 27B at 17.7 (mean over 1,000 rounds) | mixed / see build | MLX 4-bit (mlx-community/gemma-4-26b-a4b-it-4bit) | mlx-lm (MLX 0.32.x) | 85.08 | context: short prompt (~250-410 tokens), 1,000 consecutive rounds📏 1,000 Rewrites Later dataset (CC BY 4.0): the model rewrote its own previous output 1,000 times in a row (Gettysburg Address seed, ~250-410 prompt tokens, up to ~360 output tokens per round, temp 0.7, top_p 0.95, seed 42, thinking off); mlx-lm (MLX 0.32.x) in-process, weights fully in memory, stock wired-memory limit; listed figure is the mean generation tok/s over all 1,000 rounds as reported by the runtime (dataset summary), measured 23 Oct 2026; median 85.47 tok/s; MLX peak memory 14.69 GB; 84.1 mean in a second 1,000-round run at temp 1.0 |
| Inkling 2 Mini | Mac mini M5 Pro 64GB (llama.cpp IQ4_NL): Inkling 2 Mini at 29.95 tok/s short context, 17.39 tok/s at 128k (pratikgx) | mixed / see build | ggml-org IQ4_NL (17.63 GB; 4-bit experts in layers 0-19, Q8_0 elsewhere, loop controller F32) | llama.cpp PR #32694 (d7cb78c91, build 12587), Metal | 29.95 | context: 512-token prompt, 256 generated📏 Independent measurement by pratikgx, 20 Nov 2026. Median of 3 timed runs after 1 warmup; largest generation-speed spread at any context was 0.81%. Peak RSS 23.10 GB includes a reserved 128k KV slot, so it is not a tight short-context footprint. Time to first token from the streaming /completion endpoint. |
| Inkling 2 Mini | Mac mini M5 Pro 64GB (llama.cpp IQ4_NL): Inkling 2 Mini at 29.95 tok/s short context, 17.39 tok/s at 128k (pratikgx) | mixed / see build | ggml-org IQ4_NL (17.63 GB; 4-bit experts in layers 0-19, Q8_0 elsewhere, loop controller F32) | llama.cpp PR #32694 (d7cb78c91, build 12587), Metal | 28.42 | context: 4,096-token prompt, 256 generated📏 Same machine, runtime and settings as the 512-token row. Median of 3 timed runs. Peak RSS 23.53 GB (includes reserved 128k KV slot). |
| Inkling 2 Mini | Mac mini M5 Pro 64GB (llama.cpp IQ4_NL): Inkling 2 Mini at 29.95 tok/s short context, 17.39 tok/s at 128k (pratikgx) | mixed / see build | ggml-org IQ4_NL (17.63 GB; 4-bit experts in layers 0-19, Q8_0 elsewhere, loop controller F32) | llama.cpp PR #32694 (d7cb78c91, build 12587), Metal | 26.74 | context: 16,384-token prompt, 256 generated📏 Same machine, runtime and settings as the 512-token row. Median of 3 timed runs. Peak RSS 24.13 GB (includes reserved 128k KV slot). |
| Inkling 2 Mini | Mac mini M5 Pro 64GB (llama.cpp IQ4_NL): Inkling 2 Mini at 29.95 tok/s short context, 17.39 tok/s at 128k (pratikgx) | mixed / see build | ggml-org IQ4_NL (17.63 GB; 4-bit experts in layers 0-19, Q8_0 elsewhere, loop controller F32) | llama.cpp PR #32694 (d7cb78c91, build 12587), Metal | 24.97 | context: 32,768-token prompt, 256 generated📏 Same machine, runtime and settings as the 512-token row. Median of 3 timed runs. Peak RSS 24.54 GB (includes reserved 128k KV slot). |
| Inkling 2 Mini | Mac mini M5 Pro 64GB (llama.cpp IQ4_NL): Inkling 2 Mini at 29.95 tok/s short context, 17.39 tok/s at 128k (pratikgx) | mixed / see build | ggml-org IQ4_NL (17.63 GB; 4-bit experts in layers 0-19, Q8_0 elsewhere, loop controller F32) | llama.cpp PR #32694 (d7cb78c91, build 12587), Metal | 17.39 | context: 131,072-token prompt, 256 generated📏 Same machine, runtime and settings as the 512-token row. Median of 3 timed runs. The model stayed in memory. Peak RSS 25.86 GB (includes reserved 128k KV slot). TTFT 1,376 s (22.9 min). |
| Qwen3.6 35B-A3B | Mac mini M5 Pro 64GB (MLX 4-bit): Qwen3.6 35B-A3B at 100.2 tok/s, Gemma 4 26B-A4B at 85.1, Qwen3.8 27B at 17.7 (mean over 1,000 rounds) | mixed / see build | MLX 4-bit (mlx-community/Qwen3.6-35B-A3B-4bit) | mlx-lm (MLX 0.32.x) | 100.24 | context: short prompt (~250-410 tokens), 1,000 consecutive rounds📏 1,000 Rewrites Later dataset (CC BY 4.0): the model rewrote its own previous output 1,000 times in a row (Gettysburg Address seed, ~250-410 prompt tokens, up to ~360 output tokens per round, temp 0.7, top_p 0.95, seed 42, thinking off); mlx-lm (MLX 0.32.x) in-process, weights fully in memory, stock wired-memory limit; listed figure is the mean generation tok/s over all 1,000 rounds as reported by the runtime (dataset summary), measured 23 Oct 2026; median 100.41 tok/s; MLX peak memory 20.15 GB; 90.19 mean in a second 1,000-round run at temp 1.0 |
| Qwen3.8 27B | Mac mini M5 Pro 64GB (MLX 4-bit): Qwen3.6 35B-A3B at 100.2 tok/s, Gemma 4 26B-A4B at 85.1, Qwen3.8 27B at 17.7 (mean over 1,000 rounds) | mixed / see build | MLX 4-bit (mlx-community/Qwen3.8-27B-4bit) | mlx-lm (MLX 0.32.x) | 17.7 | context: short prompt (~250-410 tokens), 1,000 consecutive rounds📏 1,000 Rewrites Later dataset (CC BY 4.0): the model rewrote its own previous output 1,000 times in a row (Gettysburg Address seed, ~250-410 prompt tokens, up to ~360 output tokens per round, temp 0.7, top_p 0.95, seed 42, thinking off); mlx-lm (MLX 0.32.x) in-process, weights fully in memory, stock wired-memory limit; listed figure is the mean generation tok/s over all 1,000 rounds as reported by the runtime (dataset summary), measured 23 Oct 2026; median 17.81 tok/s; MLX peak memory 16.15 GB; 3.0 h of generation, 180,209 tokens; per-round prefill was ~350-405 tok/s on these short prompts |
| Qwen3.8 Flash-Next | MacBook Pro M5 Pro 24 GB: Qwen3.8 Flash-Next (125B-A6B) IQ3_XXS at 14.7 tok/s with experts streamed from SSD (MoEShift) | mixed / see build | ISTA-DASLab GSQ-RCO IQ3_XXS (~76 GB, 2 shards), experts streamed from the internal SSD | MoEShift 0.1.0-alpha.3 (llama.cpp-based, Metal) | 14.69 | context: 72-token prompt, 512 generated (64K ctx configured)📏 moeshift-community-v1 protocol: median of 3 measured runs after 1 warm-up, 512 output tokens, prompt cache off (cache_n 0); short-prompt generation range 13.95-14.87; peak process footprint ~8.5 GB |
| Qwen3.8 Flash-Next | MacBook Pro M5 Pro 24 GB: Qwen3.8 Flash-Next (125B-A6B) IQ3_XXS at 14.7 tok/s with experts streamed from SSD (MoEShift) | mixed / see build | ISTA-DASLab GSQ-RCO IQ3_XXS (~76 GB, 2 shards), experts streamed from the internal SSD | MoEShift 0.1.0-alpha.3 (llama.cpp-based, Metal) | 13.14 | context: 6,060-token prompt, 512 generated📏 moeshift-community-v1 protocol: median of 3 measured runs after 1 warm-up, 512 output tokens, prompt cache off (cache_n 0); long-prompt generation range 11.57-13.55, prefill 359.9-375.7 |
All Apple M5 Pro builds: see also all GPUs · best LLMs by memory size · download the dataset.
FAQ
How fast is the Apple M5 Pro for local LLMs?
The fastest plain-decode result we have is 100.24 tok/s on Qwen3.6 35B-A3B (MLX 4-bit (mlx-community/Qwen3.6-35B-A3B-4bit), mlx-lm (MLX 0.32.x)) in Mac mini M5 Pro 64GB (MLX 4-bit): Qwen3.6 35B-A3B at 100.2 tok/s, Gemma 4 26B-A4B at 85.1, Qwen3.8 27B at 17.7 (mean over 1,000 rounds). Speed depends heavily on the model, quant and engine, so compare like with like in the table. RigBase lists 10 measured results from 3 builds.
How much memory does the Apple M5 Pro have?
unified memory: 24–64 GB in our builds.
Which models have been measured on the Apple M5 Pro?
Gemma 4 26B-A4B, Inkling 2 Mini, Qwen3.6 35B-A3B, Qwen3.8 27B, Qwen3.8 Flash-Next.
Where do these numbers come from?
Every row links to a build page with its original public source (benchmark logs, READMEs, forum posts). RigBase does not re-run the benchmarks. Results marked ⚡ use speculative decoding and are not comparable 1:1 with plain decode.
Referral link: if you sign up with Google and add $10, Runpod gives you and RigBase bonus credit (terms). It never affects our data.