<!-- Markdown mirror of https://getrigbase.com/models/mistral-small-3-2-24b/ . Source of truth is the HTML page. Data licence: CC BY 4.0, please cite RigBase (https://getrigbase.com/models/mistral-small-3-2-24b/). -->
Source: https://getrigbase.com/models/mistral-small-3-2-24b/
Last updated: 2026-11-15

[Models](https://getrigbase.com/models/) / Mistral Small 3.2 24B

# Best home rig for Mistral Small 3.2 24B

Quick answer

**How much VRAM does Mistral Small 3.2 24B need?**
The weights are about 14.7 GB at Q4_K_M (24B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is RTX Spark N1X (82 GB) at 18.5 tok/s.

**What is the fastest measured hardware for Mistral Small 3.2 24B?**
RTX Spark N1X: 18.5 tok/s generation (llama.cpp b12110 (CUDA, Windows on Arm), Q4_K_M (14.3 GB)), from https://dev.to/maruhana/a-120b-model-ran-about-4x-faster-than-a-27b-one-8-local-llms-on-a-surface-laptop-ultra-5cpg.

Last updated: 15 Nov 2026

Legacy model 24B dense 1 measured result fastest plain decode: 18.5 tok/s  [model info ↗](https://huggingface.co/mistralai/Mistral-Small-3.2-24B-Instruct-2506)

Older model. For current picks see [current models](https://getrigbase.com/models/).

Real, measured generation speeds for Mistral Small 3.2 24B on home hardware, fastest first. Different quants/engines aren't directly comparable.

| Build | Quant | Engine | tok/s | Prompt processing (tok/s) | Cost | Source |
|---|---|---|---|---|---|---|
| [Surface Laptop Ultra (RTX Spark N1X, 128 GB): 8 models on llama.cpp b12110, Flint 104.8 tok/s, gpt-oss 120B 64.2, Qwen3.8 27B 16.5 (maruhana)](https://getrigbase.com/builds/surface-laptop-ultra-rtx-spark-n1x-8-models-llamacpp-maruhana/) | Q4_K_M (14.3 GB) | llama.cpp b12110 (CUDA, Windows on Arm) | **18.5** | 1,183 @512 · est. TTFT @8k ≈6.9 s* |  | [dev.to ↗](https://dev.to/maruhana/a-120b-model-ran-about-4x-faster-than-a-27b-one-8-local-llms-on-a-surface-laptop-ultra-5cpg) |

Prompt processing = prefill speed reported by the source, @ prompt length. * est. TTFT @8k is calculated from measured prefill, not measured. [Why it matters](https://getrigbase.com/can-it-run/#prefill).
Ad · this space is available **Sponsor this spot: reach people building local AI rigs.** One clearly-labeled slot per page, no tracking scripts.  [Sponsor this spot →](https://getrigbase.com/advertise/#inquiry)

**Rent before you buy:** try this model on a cloud GPU for a few dollars first: [Runpod](https://runpod.io?ref=77qcpl2c) (referral link; sign up with Google to get the new-user bonus)

## Memory math: how much VRAM / RAM does Mistral Small 3.2 24B need?

Mistral Small 3.2 24B has **24B parameters** ([source](https://huggingface.co/mistralai/Mistral-Small-3.2-24B-Instruct-2506): Mistral model card (Instruct 2506), 24B dense). Weight size = parameters × bits-per-weight ÷ 8, before context (KV cache) and runtime overhead.

| Format | Bits/weight | Weights | Smallest common memory size that holds the weights alone |
|---|---|---|---|
| Q8_0 | 8.50 | 25.5 GB | 32 GB |
| Q6_K | 6.56 | 19.7 GB | 24 GB |
| Q5_K_M | 5.70 | 17.1 GB | 24 GB |
| Q4_K_M | 4.89 | **14.7 GB** | 16 GB |
| Q3_K_M | 4.00 | 12.0 GB | 12 GB |
| Q2_K | 3.16 | 9.5 GB | 12 GB |

Bits-per-weight for GGUF quants are llama.cpp's own figures ([llama.cpp quantize README](https://github.com/ggml-org/llama.cpp/blob/master/tools/quantize/README.md)). "As measured" rows apply the source's stated bits-per-weight to the parameter count; real files differ slightly because some layers stay at higher precision. Try other combinations in the [VRAM calculator](https://getrigbase.com/can-it-run/).

## FAQ

### How much VRAM does Mistral Small 3.2 24B need?

The weights are about 14.7 GB at Q4_K_M (24B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is RTX Spark N1X (82 GB) at 18.5 tok/s.

### What is the fastest measured hardware for Mistral Small 3.2 24B?

RTX Spark N1X: 18.5 tok/s generation (llama.cpp b12110 (CUDA, Windows on Arm), Q4_K_M (14.3 GB)), from https://dev.to/maruhana/a-120b-model-ran-about-4x-faster-than-a-27b-one-8-local-llms-on-a-surface-laptop-ultra-5cpg.

Related: [what GPU do you need for 2026 models](https://getrigbase.com/guides/what-gpu-for-qwen3-6-gemma-4-gpt-oss/) · [all builds](https://getrigbase.com/builds/)
