More
Submit build

Apple M5 Pro for local LLMs

Quick answer

How fast is the Apple M5 Pro for local LLMs?
The fastest plain-decode result we have is 100.24 tok/s on Qwen3.6 35B-A3B (MLX 4-bit (mlx-community/Qwen3.6-35B-A3B-4bit), mlx-lm (MLX 0.32.x)) in Mac mini M5 Pro 64GB (MLX 4-bit): Qwen3.6 35B-A3B at 100.2 tok/s, Gemma 4 26B-A4B at 85.1, Qwen3.8 27B at 17.7 (mean over 1,000 rounds). Speed depends heavily on the model, quant and engine, so compare like with like in the table. RigBase lists 10 measured results from 3 builds.

Last updated:

10 measured results from 3 builds, each linked to its source.

Memoryunified memory: 24–64 GB in our builds
Memory bandwidthnot listed
Typical EU pricenot tracked yet

All builds with the Apple M5 Pro

BuildGPU(s)Memory GB (total)Results
Mac mini M5 Pro 64GB (MLX 4-bit): Qwen3.6 35B-A3B at 100.2 tok/s, Gemma 4 26B-A4B at 85.1, Qwen3.8 27B at 17.7 (mean over 1,000 rounds)Apple M5 Pro (unified, Mac mini)643
Mac mini M5 Pro 64GB (llama.cpp IQ4_NL): Inkling 2 Mini at 29.95 tok/s short context, 17.39 tok/s at 128k (pratikgx)Apple M5 Pro (unified, Mac mini)645
MacBook Pro M5 Pro 24 GB: Qwen3.8 Flash-Next (125B-A6B) IQ3_XXS at 14.7 tok/s with experts streamed from SSD (MoEShift)Apple M5 Pro (unified)242

All measured results

(MTP, DFlash, DSpark and similar draft-and-verify methods give much higher tok/s on predictable text and are not comparable 1:1 with plain decode)

ModelBuildRan onQuantEnginetok/sDetails
Gemma 4 26B-A4BMac mini M5 Pro 64GB (MLX 4-bit): Qwen3.6 35B-A3B at 100.2 tok/s, Gemma 4 26B-A4B at 85.1, Qwen3.8 27B at 17.7 (mean over 1,000 rounds)mixed / see buildMLX 4-bit (mlx-community/gemma-4-26b-a4b-it-4bit)mlx-lm (MLX 0.32.x)85.08context: short prompt (~250-410 tokens), 1,000 consecutive rounds📏 1,000 Rewrites Later dataset (CC BY 4.0): the model rewrote its own previous output 1,000 times in a row (Gettysburg Address seed, ~250-410 prompt tokens, up to ~360 output tokens per round, temp 0.7, top_p 0.95, seed 42, thinking off); mlx-lm (MLX 0.32.x) in-process, weights fully in memory, stock wired-memory limit; listed figure is the mean generation tok/s over all 1,000 rounds as reported by the runtime (dataset summary), measured 23 Oct 2026; median 85.47 tok/s; MLX peak memory 14.69 GB; 84.1 mean in a second 1,000-round run at temp 1.0
Inkling 2 MiniMac mini M5 Pro 64GB (llama.cpp IQ4_NL): Inkling 2 Mini at 29.95 tok/s short context, 17.39 tok/s at 128k (pratikgx)mixed / see buildggml-org IQ4_NL (17.63 GB; 4-bit experts in layers 0-19, Q8_0 elsewhere, loop controller F32)llama.cpp PR #32694 (d7cb78c91, build 12587), Metal29.95context: 512-token prompt, 256 generated📏 Independent measurement by pratikgx, 20 Nov 2026. Median of 3 timed runs after 1 warmup; largest generation-speed spread at any context was 0.81%. Peak RSS 23.10 GB includes a reserved 128k KV slot, so it is not a tight short-context footprint. Time to first token from the streaming /completion endpoint.
Inkling 2 MiniMac mini M5 Pro 64GB (llama.cpp IQ4_NL): Inkling 2 Mini at 29.95 tok/s short context, 17.39 tok/s at 128k (pratikgx)mixed / see buildggml-org IQ4_NL (17.63 GB; 4-bit experts in layers 0-19, Q8_0 elsewhere, loop controller F32)llama.cpp PR #32694 (d7cb78c91, build 12587), Metal28.42context: 4,096-token prompt, 256 generated📏 Same machine, runtime and settings as the 512-token row. Median of 3 timed runs. Peak RSS 23.53 GB (includes reserved 128k KV slot).
Inkling 2 MiniMac mini M5 Pro 64GB (llama.cpp IQ4_NL): Inkling 2 Mini at 29.95 tok/s short context, 17.39 tok/s at 128k (pratikgx)mixed / see buildggml-org IQ4_NL (17.63 GB; 4-bit experts in layers 0-19, Q8_0 elsewhere, loop controller F32)llama.cpp PR #32694 (d7cb78c91, build 12587), Metal26.74context: 16,384-token prompt, 256 generated📏 Same machine, runtime and settings as the 512-token row. Median of 3 timed runs. Peak RSS 24.13 GB (includes reserved 128k KV slot).
Inkling 2 MiniMac mini M5 Pro 64GB (llama.cpp IQ4_NL): Inkling 2 Mini at 29.95 tok/s short context, 17.39 tok/s at 128k (pratikgx)mixed / see buildggml-org IQ4_NL (17.63 GB; 4-bit experts in layers 0-19, Q8_0 elsewhere, loop controller F32)llama.cpp PR #32694 (d7cb78c91, build 12587), Metal24.97context: 32,768-token prompt, 256 generated📏 Same machine, runtime and settings as the 512-token row. Median of 3 timed runs. Peak RSS 24.54 GB (includes reserved 128k KV slot).
Inkling 2 MiniMac mini M5 Pro 64GB (llama.cpp IQ4_NL): Inkling 2 Mini at 29.95 tok/s short context, 17.39 tok/s at 128k (pratikgx)mixed / see buildggml-org IQ4_NL (17.63 GB; 4-bit experts in layers 0-19, Q8_0 elsewhere, loop controller F32)llama.cpp PR #32694 (d7cb78c91, build 12587), Metal17.39context: 131,072-token prompt, 256 generated📏 Same machine, runtime and settings as the 512-token row. Median of 3 timed runs. The model stayed in memory. Peak RSS 25.86 GB (includes reserved 128k KV slot). TTFT 1,376 s (22.9 min).
Qwen3.6 35B-A3BMac mini M5 Pro 64GB (MLX 4-bit): Qwen3.6 35B-A3B at 100.2 tok/s, Gemma 4 26B-A4B at 85.1, Qwen3.8 27B at 17.7 (mean over 1,000 rounds)mixed / see buildMLX 4-bit (mlx-community/Qwen3.6-35B-A3B-4bit)mlx-lm (MLX 0.32.x)100.24context: short prompt (~250-410 tokens), 1,000 consecutive rounds📏 1,000 Rewrites Later dataset (CC BY 4.0): the model rewrote its own previous output 1,000 times in a row (Gettysburg Address seed, ~250-410 prompt tokens, up to ~360 output tokens per round, temp 0.7, top_p 0.95, seed 42, thinking off); mlx-lm (MLX 0.32.x) in-process, weights fully in memory, stock wired-memory limit; listed figure is the mean generation tok/s over all 1,000 rounds as reported by the runtime (dataset summary), measured 23 Oct 2026; median 100.41 tok/s; MLX peak memory 20.15 GB; 90.19 mean in a second 1,000-round run at temp 1.0
Qwen3.8 27BMac mini M5 Pro 64GB (MLX 4-bit): Qwen3.6 35B-A3B at 100.2 tok/s, Gemma 4 26B-A4B at 85.1, Qwen3.8 27B at 17.7 (mean over 1,000 rounds)mixed / see buildMLX 4-bit (mlx-community/Qwen3.8-27B-4bit)mlx-lm (MLX 0.32.x)17.7context: short prompt (~250-410 tokens), 1,000 consecutive rounds📏 1,000 Rewrites Later dataset (CC BY 4.0): the model rewrote its own previous output 1,000 times in a row (Gettysburg Address seed, ~250-410 prompt tokens, up to ~360 output tokens per round, temp 0.7, top_p 0.95, seed 42, thinking off); mlx-lm (MLX 0.32.x) in-process, weights fully in memory, stock wired-memory limit; listed figure is the mean generation tok/s over all 1,000 rounds as reported by the runtime (dataset summary), measured 23 Oct 2026; median 17.81 tok/s; MLX peak memory 16.15 GB; 3.0 h of generation, 180,209 tokens; per-round prefill was ~350-405 tok/s on these short prompts
Qwen3.8 Flash-NextMacBook Pro M5 Pro 24 GB: Qwen3.8 Flash-Next (125B-A6B) IQ3_XXS at 14.7 tok/s with experts streamed from SSD (MoEShift)mixed / see buildISTA-DASLab GSQ-RCO IQ3_XXS (~76 GB, 2 shards), experts streamed from the internal SSDMoEShift 0.1.0-alpha.3 (llama.cpp-based, Metal)14.69context: 72-token prompt, 512 generated (64K ctx configured)📏 moeshift-community-v1 protocol: median of 3 measured runs after 1 warm-up, 512 output tokens, prompt cache off (cache_n 0); short-prompt generation range 13.95-14.87; peak process footprint ~8.5 GB
Qwen3.8 Flash-NextMacBook Pro M5 Pro 24 GB: Qwen3.8 Flash-Next (125B-A6B) IQ3_XXS at 14.7 tok/s with experts streamed from SSD (MoEShift)mixed / see buildISTA-DASLab GSQ-RCO IQ3_XXS (~76 GB, 2 shards), experts streamed from the internal SSDMoEShift 0.1.0-alpha.3 (llama.cpp-based, Metal)13.14context: 6,060-token prompt, 512 generated📏 moeshift-community-v1 protocol: median of 3 measured runs after 1 warm-up, 512 output tokens, prompt cache off (cache_n 0); long-prompt generation range 11.57-13.55, prefill 359.9-375.7

All Apple M5 Pro builds: see also all GPUs · best LLMs by memory size · download the dataset.

FAQ

How fast is the Apple M5 Pro for local LLMs?

The fastest plain-decode result we have is 100.24 tok/s on Qwen3.6 35B-A3B (MLX 4-bit (mlx-community/Qwen3.6-35B-A3B-4bit), mlx-lm (MLX 0.32.x)) in Mac mini M5 Pro 64GB (MLX 4-bit): Qwen3.6 35B-A3B at 100.2 tok/s, Gemma 4 26B-A4B at 85.1, Qwen3.8 27B at 17.7 (mean over 1,000 rounds). Speed depends heavily on the model, quant and engine, so compare like with like in the table. RigBase lists 10 measured results from 3 builds.

How much memory does the Apple M5 Pro have?

unified memory: 24–64 GB in our builds.

Which models have been measured on the Apple M5 Pro?

Gemma 4 26B-A4B, Inkling 2 Mini, Qwen3.6 35B-A3B, Qwen3.8 27B, Qwen3.8 Flash-Next.

Where do these numbers come from?

Every row links to a build page with its original public source (benchmark logs, READMEs, forum posts). RigBase does not re-run the benchmarks. Results marked ⚡ use speculative decoding and are not comparable 1:1 with plain decode.

Not enough VRAM? Rent before you buy. Run Apple M5 Pro-class models on a rented 24–96 GB cloud GPU by the hour and see the real speed before spending €1,000+. Try it on Runpod
Referral link: if you sign up with Google and add $10, Runpod gives you and RigBase bonus credit (terms). It never affects our data.