More
Submit build

Models / Mistral Small 3.2 24B

Best home rig for Mistral Small 3.2 24B

Quick answer

How much VRAM does Mistral Small 3.2 24B need?
The weights are about 14.7 GB at Q4_K_M (24B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is RTX Spark N1X (82 GB) at 18.5 tok/s.

What is the fastest measured hardware for Mistral Small 3.2 24B?
RTX Spark N1X: 18.5 tok/s generation (llama.cpp b12110 (CUDA, Windows on Arm), Q4_K_M (14.3 GB)), from https://dev.to/maruhana/a-120b-model-ran-about-4x-faster-than-a-27b-one-8-local-llms-on-a-surface-laptop-ultra-5cpg.

Last updated:

Legacy model24B dense1 measured resultfastest plain decode: 18.5 tok/s model info ↗

Older model. For current picks see current models.

Real, measured generation speeds for Mistral Small 3.2 24B on home hardware, fastest first. Different quants/engines aren't directly comparable.

BuildQuantEnginetok/sPrompt processing (tok/s)CostSource
Surface Laptop Ultra (RTX Spark N1X, 128 GB): 8 models on llama.cpp b12110, Flint 104.8 tok/s, gpt-oss 120B 64.2, Qwen3.8 27B 16.5 (maruhana)Q4_K_M (14.3 GB)llama.cpp b12110 (CUDA, Windows on Arm)18.51,183 @512 · est. TTFT @8k ≈6.9 s*dev.to ↗

Prompt processing = prefill speed reported by the source, @ prompt length. * est. TTFT @8k is calculated from measured prefill, not measured. Why it matters.

Rent before you buy: try this model on a cloud GPU for a few dollars first: Runpod (referral link; sign up with Google to get the new-user bonus)

Memory math: how much VRAM / RAM does Mistral Small 3.2 24B need?

Mistral Small 3.2 24B has 24B parameters (source: Mistral model card (Instruct 2506), 24B dense). Weight size = parameters × bits-per-weight ÷ 8, before context (KV cache) and runtime overhead.

FormatBits/weightWeightsSmallest common memory size that holds the weights alone
Q8_08.5025.5 GB32 GB
Q6_K6.5619.7 GB24 GB
Q5_K_M5.7017.1 GB24 GB
Q4_K_M4.8914.7 GB16 GB
Q3_K_M4.0012.0 GB12 GB
Q2_K3.169.5 GB12 GB

Bits-per-weight for GGUF quants are llama.cpp's own figures (llama.cpp quantize README). "As measured" rows apply the source's stated bits-per-weight to the parameter count; real files differ slightly because some layers stay at higher precision. Try other combinations in the VRAM calculator.

FAQ

How much VRAM does Mistral Small 3.2 24B need?

The weights are about 14.7 GB at Q4_K_M (24B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is RTX Spark N1X (82 GB) at 18.5 tok/s.

What is the fastest measured hardware for Mistral Small 3.2 24B?

RTX Spark N1X: 18.5 tok/s generation (llama.cpp b12110 (CUDA, Windows on Arm), Q4_K_M (14.3 GB)), from https://dev.to/maruhana/a-120b-model-ran-about-4x-faster-than-a-27b-one-8-local-llms-on-a-surface-laptop-ultra-5cpg.

Related: what GPU do you need for 2026 models · all builds