<!-- Markdown mirror of https://getrigbase.com/models/inkling-2-mini/ . Source of truth is the HTML page. Data licence: CC BY 4.0, please cite RigBase (https://getrigbase.com/models/inkling-2-mini/). -->
Source: https://getrigbase.com/models/inkling-2-mini/
Last updated: 2026-11-22

[Models](https://getrigbase.com/models/) / Inkling 2 Mini

# Best home rig for Inkling 2 Mini

Quick answer

**How much VRAM does Inkling 2 Mini need?**
The weights are about 20 GB in its native format (Thinking Machines Lab model card: 26.4B total / 3.4B active looped MoE (middle layers run 3 times, ~5.4B effective active), text + image in, up to 1M context, Apache 2.0, released 20 Nov 2026; one ~20 GB NVFP4/MXFP8 checkpoint). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is RTX 3090 (24 GB) at 73.5 tok/s.

**What is the fastest measured hardware for Inkling 2 Mini?**
2x RTX 3090: 136.6 tok/s generation (vLLM TP2 + Ampere overlay, NVLink, 1 stream, W4A16 INT4 + DSpark), from https://github.com/tonyd2wild/Inkling-2-Mini-W4A16-RTX3090.

Last updated: 22 Nov 2026

Current model 26.4B total · 3.4B active 31 measured results fastest plain decode: 59.7 tok/s  [model info ↗](https://huggingface.co/thinkingmachines/Inkling-2-Mini)

[Will it fit my GPU?](https://getrigbase.com/can-it-run/) [Best value GPU per euro →](https://getrigbase.com/guides/best-value-gpu-local-llm-europe/)

Real, measured generation speeds for Inkling 2 Mini on home hardware, fastest first. Different quants/engines aren't directly comparable.

| Build | Quant | Engine | tok/s | Prompt processing (tok/s) | Cost | Source |
|---|---|---|---|---|---|---|
| [2x RTX 3090: Inkling 2 Mini W4A16 INT4 on vLLM TP2 + Ampere overlay, 136.6 tok/s bench (Tech2wild)](https://getrigbase.com/builds/tony-2x-rtx-3090-inkling-2-mini-w4a16-rtx3090/) | W4A16 INT4 + DSpark | vLLM TP2 + Ampere overlay, NVLink, 1 stream | **136.6** ⚡ | – |  | [github.com ↗](https://github.com/tonyd2wild/Inkling-2-Mini-W4A16-RTX3090) |
| [2x RTX 3090: Inkling 2 Mini W4A16 INT4 on vLLM TP2 + Ampere overlay, 136.6 tok/s bench (Tech2wild)](https://getrigbase.com/builds/tony-2x-rtx-3090-inkling-2-mini-w4a16-rtx3090/) | W4A16 INT4 + DSpark | vLLM TP2 + Ampere overlay, NVLink, 1 stream | **119** ⚡ | 2,302 @439,384 · est. TTFT @8k ≈3.6 s* |  | [github.com ↗](https://github.com/tonyd2wild/Inkling-2-Mini-W4A16-RTX3090) |
| [2x RTX 3090: Inkling 2 Mini W4A16 INT4 on vLLM TP2 + Ampere overlay, 136.6 tok/s bench (Tech2wild)](https://getrigbase.com/builds/tony-2x-rtx-3090-inkling-2-mini-w4a16-rtx3090/) | W4A16 INT4 + DSpark | vLLM TP2 + Ampere overlay, NVLink P2P off (NCCL_P2P_DISABLE=1), 1 stream | **100.5** ⚡ | – |  | [github.com ↗](https://github.com/tonyd2wild/Inkling-2-Mini-W4A16-RTX3090) |
| [2x RTX 3090: Inkling 2 Mini W4A16 INT4 on vLLM TP2 + Ampere overlay, 136.6 tok/s bench (Tech2wild)](https://getrigbase.com/builds/tony-2x-rtx-3090-inkling-2-mini-w4a16-rtx3090/) | W4A16 INT4 | vLLM TP2 + Ampere overlay, 1 stream | **89.8** ⚡ | – |  | [github.com ↗](https://github.com/tonyd2wild/Inkling-2-Mini-W4A16-RTX3090) |
| [Mac Studio M5 Ultra 96GB (oMLX oQ8e + Lightning MTP): Inkling 2 Mini at 83.4 tok/s at 4k context, 56.2 tok/s at 32k](https://getrigbase.com/builds/omlx-m5-ultra-96gb-inkling-2-mini-oq8e-mtp/) | oMLX oQ8e, Affine8 8-bit KV | oMLX v0.8.1, Lightning MTP | **83.4** ⚡ | 3,225 @4k · est. TTFT @8k ≈2.5 s* |  | [omlx.ai ↗](https://omlx.ai/benchmarks/performance/t3gjl6vk) |
| [Mac Studio M5 Ultra 96GB (oMLX oQ8e + Lightning MTP): Inkling 2 Mini at 83.4 tok/s at 4k context, 56.2 tok/s at 32k](https://getrigbase.com/builds/omlx-m5-ultra-96gb-inkling-2-mini-oq8e-mtp/) | oMLX oQ8e, Affine8 8-bit KV | oMLX v0.8.1, Lightning MTP | **76.6** ⚡ | 2,547 @1k · est. TTFT @8k ≈3.2 s* |  | [omlx.ai ↗](https://omlx.ai/benchmarks/performance/t3gjl6vk) |
| [Mac Studio M5 Ultra 96GB (oMLX oQ8e + Lightning MTP): Inkling 2 Mini at 83.4 tok/s at 4k context, 56.2 tok/s at 32k](https://getrigbase.com/builds/omlx-m5-ultra-96gb-inkling-2-mini-oq8e-mtp/) | oMLX oQ8e, Affine8 8-bit KV | oMLX v0.8.1, Lightning MTP | **74.7** ⚡ | 1,770 @128k · est. TTFT @8k ≈4.6 s* |  | [omlx.ai ↗](https://omlx.ai/benchmarks/performance/t3gjl6vk) |
| [RTX 3090: Inkling 2 Mini W4A16 INT4 on vLLM TP1 + Ampere overlay, 73.5 tok/s median (Tech2wild)](https://getrigbase.com/builds/tony-rtx-3090-inkling-2-mini-w4a16-rtx3090/) | W4A16 INT4 | vLLM TP1 + Ampere overlay, 1 stream | **73.5** ⚡ | – |  | [github.com ↗](https://github.com/tonyd2wild/Inkling-2-Mini-W4A16-RTX3090) |
| [RTX 4090: Inkling 2 Mini at 59.7 tok/s decode and 1,162 tok/s prefill (8K prompt) on the Xolotl engine, 14 GB XQ quant vs 57 tok/s for the exact 22.9 GB conversion](https://getrigbase.com/builds/rtx-4090-inkling-2-mini-xq-14gb-xolotl-engine-indrasmirror/) | XQ mixed (~4.0 bpw routed experts, 13 of 24 layers keep official NVFP4; Q8_0 backbone; 13.93 GB), abliterated | Xolotl (xolotl-inkling) | **59.7** | 1,162 @8K · est. TTFT @8k ≈7.0 s* |  | [huggingface.co ↗](https://huggingface.co/IndrasMirror/Inkling-2-Mini-XQ-Uncensored) |
| [RTX 4090: Inkling 2 Mini at 59.7 tok/s decode and 1,162 tok/s prefill (8K prompt) on the Xolotl engine, 14 GB XQ quant vs 57 tok/s for the exact 22.9 GB conversion](https://getrigbase.com/builds/rtx-4090-inkling-2-mini-xq-14gb-xolotl-engine-indrasmirror/) | Official precision, exact conversion (NVFP4 + MXFP8 experts, BF16 rest; 22.9 GB) | Xolotl (xolotl-inkling) | **57** | 1,131 @8K · est. TTFT @8k ≈7.2 s* |  | [huggingface.co ↗](https://huggingface.co/IndrasMirror/Inkling-2-Mini-XQ-Uncensored) |
| [Mac Studio M5 Ultra 96GB (oMLX oQ8e + Lightning MTP): Inkling 2 Mini at 83.4 tok/s at 4k context, 56.2 tok/s at 32k](https://getrigbase.com/builds/omlx-m5-ultra-96gb-inkling-2-mini-oq8e-mtp/) | oMLX oQ8e, Affine8 8-bit KV | oMLX v0.8.1, Lightning MTP | **56.2** ⚡ | 2,968 @32k · est. TTFT @8k ≈2.8 s* |  | [omlx.ai ↗](https://omlx.ai/benchmarks/performance/t3gjl6vk) |
| [Mac Studio M5 Ultra 96GB (oMLX oQ8e + Lightning MTP): Inkling 2 Mini at 83.4 tok/s at 4k context, 56.2 tok/s at 32k](https://getrigbase.com/builds/omlx-m5-ultra-96gb-inkling-2-mini-oq8e-mtp/) | oMLX oQ8e, Affine8 8-bit KV | oMLX v0.8.1, Lightning MTP | **55.4** ⚡ | 1,164 @195k · est. TTFT @8k ≈7.0 s* |  | [omlx.ai ↗](https://omlx.ai/benchmarks/performance/t3gjl6vk) |
| [DGX Spark: Inkling 2 Mini NVFP4/MXFP8 on vLLM TP1, 47.7 tok/s eval (Tech2wild)](https://getrigbase.com/builds/tony-dgx-spark-inkling-2-mini-nvfp4-dgx-spark/) | NVFP4/MXFP8 + DSpark | vLLM TP1, 1 stream | **47.7** ⚡ | – |  | [github.com ↗](https://github.com/tonyd2wild/Inkling-2-Mini-NVFP4-DGX-Spark) |
| [DGX Spark: Inkling 2 Mini NVFP4/MXFP8 on vLLM TP1, 47.7 tok/s eval (Tech2wild)](https://getrigbase.com/builds/tony-dgx-spark-inkling-2-mini-nvfp4-dgx-spark/) | NVFP4/MXFP8 + DSpark, FP8 KV | vLLM TP1, 1 stream | **47.3** ⚡ | – |  | [github.com ↗](https://github.com/tonyd2wild/Inkling-2-Mini-NVFP4-DGX-Spark) |
| [DGX Spark: Inkling 2 Mini NVFP4/MXFP8 on vLLM TP1, 47.7 tok/s eval (Tech2wild)](https://getrigbase.com/builds/tony-dgx-spark-inkling-2-mini-nvfp4-dgx-spark/) | NVFP4/MXFP8 + DSpark, FP8 KV | vLLM TP1, 1 stream | **47.2** ⚡ | 1,693 @199,989 · est. TTFT @8k ≈4.8 s* |  | [github.com ↗](https://github.com/tonyd2wild/Inkling-2-Mini-NVFP4-DGX-Spark) |
| [Mac Studio M3 Ultra 512GB (oMLX, MTP off): Inkling 2 Mini at 40.3 tok/s at 1k context, 28.4 tok/s at 195k](https://getrigbase.com/builds/omlx-m3-ultra-512gb-inkling-2-mini/) | oMLX Inkling-2-Mini (as labelled) | oMLX v0.8.1, MTP off | **40.3** | 1,169 @1k · est. TTFT @8k ≈7.0 s* |  | [omlx.ai ↗](https://omlx.ai/benchmarks/performance/z2wrhp3o) |
| [Mac Studio M3 Ultra 512GB (oMLX, MTP off): Inkling 2 Mini at 40.3 tok/s at 1k context, 28.4 tok/s at 195k](https://getrigbase.com/builds/omlx-m3-ultra-512gb-inkling-2-mini/) | oMLX Inkling-2-Mini (as labelled) | oMLX v0.8.1, MTP off | **39.5** | 1,367 @4k · est. TTFT @8k ≈6.0 s* |  | [omlx.ai ↗](https://omlx.ai/benchmarks/performance/z2wrhp3o) |
| [Mac Studio M3 Ultra 512GB (oMLX, MTP off): Inkling 2 Mini at 40.3 tok/s at 1k context, 28.4 tok/s at 195k](https://getrigbase.com/builds/omlx-m3-ultra-512gb-inkling-2-mini/) | oMLX Inkling-2-Mini (as labelled) | oMLX v0.8.1, MTP off | **39.4** | 1,360 @8k · est. TTFT @8k ≈6.0 s* |  | [omlx.ai ↗](https://omlx.ai/benchmarks/performance/z2wrhp3o) |
| [Mac Studio M3 Ultra 512GB (oMLX, MTP off): Inkling 2 Mini at 40.3 tok/s at 1k context, 28.4 tok/s at 195k](https://getrigbase.com/builds/omlx-m3-ultra-512gb-inkling-2-mini/) | oMLX Inkling-2-Mini (as labelled) | oMLX v0.8.1, MTP off | **38.7** | 1,328 @16k · est. TTFT @8k ≈6.2 s* |  | [omlx.ai ↗](https://omlx.ai/benchmarks/performance/z2wrhp3o) |
| [Mac Studio M3 Ultra 512GB (oMLX, MTP off): Inkling 2 Mini at 40.3 tok/s at 1k context, 28.4 tok/s at 195k](https://getrigbase.com/builds/omlx-m3-ultra-512gb-inkling-2-mini/) | oMLX Inkling-2-Mini (as labelled) | oMLX v0.8.1, MTP off | **37.3** | 1,256 @32k · est. TTFT @8k ≈6.5 s* |  | [omlx.ai ↗](https://omlx.ai/benchmarks/performance/z2wrhp3o) |
| [Mac Studio M3 Ultra 512GB (oMLX, MTP off): Inkling 2 Mini at 40.3 tok/s at 1k context, 28.4 tok/s at 195k](https://getrigbase.com/builds/omlx-m3-ultra-512gb-inkling-2-mini/) | oMLX Inkling-2-Mini (as labelled) | oMLX v0.8.1, MTP off | **34.4** | 1,130 @64k · est. TTFT @8k ≈7.2 s* |  | [omlx.ai ↗](https://omlx.ai/benchmarks/performance/z2wrhp3o) |
| [Mac Studio M3 Ultra 512GB (oMLX, MTP off): Inkling 2 Mini at 40.3 tok/s at 1k context, 28.4 tok/s at 195k](https://getrigbase.com/builds/omlx-m3-ultra-512gb-inkling-2-mini/) | oMLX Inkling-2-Mini (as labelled) | oMLX v0.8.1, MTP off | **30.9** | 941 @128k · est. TTFT @8k ≈8.7 s* |  | [omlx.ai ↗](https://omlx.ai/benchmarks/performance/z2wrhp3o) |
| [Mac mini M5 Pro 64GB (llama.cpp IQ4_NL): Inkling 2 Mini at 29.95 tok/s short context, 17.39 tok/s at 128k (pratikgx)](https://getrigbase.com/builds/mac-mini-m5-pro-64gb-inkling-2-mini-iq4nl-llamacpp-pratikgx/) | ggml-org IQ4_NL (17.63 GB; 4-bit experts in layers 0-19, Q8_0 elsewhere, loop controller F32) | llama.cpp PR #32694 (d7cb78c91, build 12587), Metal | **29.95** | 491 @512 · est. TTFT @8k ≈16.7 s* |  | [huggingface.co ↗](https://huggingface.co/datasets/pratikgx/inkling-2-mini-m5-pro-mac-mini-bench) |
| [Mac mini M5 Pro 64GB (llama.cpp IQ4_NL): Inkling 2 Mini at 29.95 tok/s short context, 17.39 tok/s at 128k (pratikgx)](https://getrigbase.com/builds/mac-mini-m5-pro-64gb-inkling-2-mini-iq4nl-llamacpp-pratikgx/) | ggml-org IQ4_NL (17.63 GB; 4-bit experts in layers 0-19, Q8_0 elsewhere, loop controller F32) | llama.cpp PR #32694 (d7cb78c91, build 12587), Metal | **28.42** | 358 @4,096 · est. TTFT @8k ≈22.9 s* |  | [huggingface.co ↗](https://huggingface.co/datasets/pratikgx/inkling-2-mini-m5-pro-mac-mini-bench) |
| [Mac Studio M3 Ultra 512GB (oMLX, MTP off): Inkling 2 Mini at 40.3 tok/s at 1k context, 28.4 tok/s at 195k](https://getrigbase.com/builds/omlx-m3-ultra-512gb-inkling-2-mini/) | oMLX Inkling-2-Mini (as labelled) | oMLX v0.8.1, MTP off | **28.4** | 801 @195k · est. TTFT @8k ≈10.2 s* |  | [omlx.ai ↗](https://omlx.ai/benchmarks/performance/z2wrhp3o) |
| [DGX Spark: Inkling 2 Mini NVFP4/MXFP8 on vLLM TP1, 47.7 tok/s eval (Tech2wild)](https://getrigbase.com/builds/tony-dgx-spark-inkling-2-mini-nvfp4-dgx-spark/) | NVFP4/MXFP8 + DSpark, BF16 KV | vLLM TP1, 1 stream | **28.3** ⚡ | 1,680 @199,989 · est. TTFT @8k ≈4.9 s* |  | [github.com ↗](https://github.com/tonyd2wild/Inkling-2-Mini-NVFP4-DGX-Spark) |
| [Mac mini M5 Pro 64GB (llama.cpp IQ4_NL): Inkling 2 Mini at 29.95 tok/s short context, 17.39 tok/s at 128k (pratikgx)](https://getrigbase.com/builds/mac-mini-m5-pro-64gb-inkling-2-mini-iq4nl-llamacpp-pratikgx/) | ggml-org IQ4_NL (17.63 GB; 4-bit experts in layers 0-19, Q8_0 elsewhere, loop controller F32) | llama.cpp PR #32694 (d7cb78c91, build 12587), Metal | **26.74** | 248 @16,384 · est. TTFT @8k ≈33.0 s* |  | [huggingface.co ↗](https://huggingface.co/datasets/pratikgx/inkling-2-mini-m5-pro-mac-mini-bench) |
| [Mac mini M5 Pro 64GB (llama.cpp IQ4_NL): Inkling 2 Mini at 29.95 tok/s short context, 17.39 tok/s at 128k (pratikgx)](https://getrigbase.com/builds/mac-mini-m5-pro-64gb-inkling-2-mini-iq4nl-llamacpp-pratikgx/) | ggml-org IQ4_NL (17.63 GB; 4-bit experts in layers 0-19, Q8_0 elsewhere, loop controller F32) | llama.cpp PR #32694 (d7cb78c91, build 12587), Metal | **24.97** | 202 @32,768 · est. TTFT @8k ≈40.6 s* |  | [huggingface.co ↗](https://huggingface.co/datasets/pratikgx/inkling-2-mini-m5-pro-mac-mini-bench) |
| [RX 9070 XT 16 GB (Windows, llama.cpp Vulkan): Inkling 2 Mini IQ3_XXS at 20.8 tok/s, slower than dense Qwen3.8 27B IQ3_XXS at 25.6; Qwen3.6 35B-A3B does 142](https://getrigbase.com/builds/rx-9070-xt-16gb-inkling-2-mini-iq3xxs-vs-qwen38-27b-vulkan-windows/) | IQ3_XXS (InterpreticAI, 10.9 GiB) | llama.cpp b12600 Vulkan (Windows) | **20.8** | 403 @2048 · est. TTFT @8k ≈20.3 s* |  | [github.com ↗](https://github.com/ggml-org/llama.cpp/issues/32707) |
| [DGX Spark: Inkling 2 Mini NVFP4/MXFP8 on vLLM TP1, 47.7 tok/s eval (Tech2wild)](https://getrigbase.com/builds/tony-dgx-spark-inkling-2-mini-nvfp4-dgx-spark/) | NVFP4/MXFP8 + DSpark, FP8 KV | vLLM TP1, 1 stream | **18.4** ⚡ | 598 @949,953 · est. TTFT @8k ≈13.7 s* |  | [github.com ↗](https://github.com/tonyd2wild/Inkling-2-Mini-NVFP4-DGX-Spark) |
| [Mac mini M5 Pro 64GB (llama.cpp IQ4_NL): Inkling 2 Mini at 29.95 tok/s short context, 17.39 tok/s at 128k (pratikgx)](https://getrigbase.com/builds/mac-mini-m5-pro-64gb-inkling-2-mini-iq4nl-llamacpp-pratikgx/) | ggml-org IQ4_NL (17.63 GB; 4-bit experts in layers 0-19, Q8_0 elsewhere, loop controller F32) | llama.cpp PR #32694 (d7cb78c91, build 12587), Metal | **17.39** | 95.29 @131,072 · est. TTFT @8k ≈86.0 s* |  | [huggingface.co ↗](https://huggingface.co/datasets/pratikgx/inkling-2-mini-m5-pro-mac-mini-bench) |

Prompt processing = prefill speed reported by the source, @ prompt length. * est. TTFT @8k is calculated from measured prefill, not measured. [Why it matters](https://getrigbase.com/can-it-run/#prefill).

## Quality cost of each quant for Inkling 2 Mini

How far each quantization drifts from the full-precision (BF16) model, as published by the quant maker. Lower KL divergence (KLD) is closer to the original; same-top-token % is how often the quant picks the same next token as BF16. Q4_K_M is usually the sweet spot; below Q4 the drift grows fast, which is why RigBase only lists Q4-or-higher speed results.

| Quant | File size | Mean KLD vs BF16 | Same top token |
|---|---|---|---|
| UD-Q4_K_XL | 17.63 GB | **0.0107** | 98.0% |
| UD-Q4_K_M | 16.61 GB | **0.0108** | 97.9% |
| UD-IQ4_XS | 15.73 GB | **0.0109** | 97.7% |
| UD-Q3_K_XL below Q4 | 13.63 GB | **0.0174** | 96.3% |
| UD-IQ3_XXS below Q4 | 11.66 GB | **0.0269** | 94.7% |
| UD-Q2_K_XL below Q4 | 9.14 GB | **0.0595** | 91.6% |
| UD-IQ2_XXS below Q4 | 8.49 GB | **0.0726** | 90.6% |
| UD-IQ1_M below Q4 | 8.02 GB | **0.0867** | 90.2% |
| UD-IQ1_S below Q4 | 7.76 GB | **0.0911** | 89.4% |

Source: [unsloth.ai ↗](https://unsloth.ai/docs/models/inkling-2-mini). Unsloth (quant publisher) run-locally guide, Nov 2026: each GGUF vs its UD-Q8_K_XL file (22.87 GB) across seven domains with identical llama.cpp settings. The reference keeps the official NVFP4 expert weights and stores the rest in BF16, i.e. the released precision, so it is the practical full-precision baseline for this model. KLD from different publishers, datasets or llama.cpp builds isn't directly comparable.
Ad · this space is available **Sponsor this spot: reach people building local AI rigs.** One clearly-labeled slot per page, no tracking scripts.  [Sponsor this spot →](https://getrigbase.com/advertise/#inquiry)

**Rent before you buy:** try this model on a cloud GPU for a few dollars first: [Runpod](https://runpod.io?ref=77qcpl2c) (referral link; sign up with Google to get the new-user bonus)

## Memory math: how much VRAM / RAM does Inkling 2 Mini need?

Inkling 2 Mini has **26.4B parameters**, about **3.4B active** per token ([source](https://huggingface.co/thinkingmachines/Inkling-2-Mini): Thinking Machines Lab model card: 26.4B total / 3.4B active looped MoE (middle layers run 3 times, ~5.4B effective active), text + image in, up to 1M context, Apache 2.0, released 20 Nov 2026; one ~20 GB NVFP4/MXFP8 checkpoint). Weight size = parameters × bits-per-weight ÷ 8, before context (KV cache) and runtime overhead.

Official checkpoint is already NVFP4/MXFP8 (~20 GB on disk); there is no separate BF16 download.

| Format | Bits/weight | Weights | Smallest common memory size that holds the weights alone |
|---|---|---|---|
| Native release format | – | **≈20 GB** | 24 GB |
| Q8_0 | 8.50 | 28.1 GB | 32 GB |
| Q6_K | 6.56 | 21.7 GB | 24 GB |
| Q5_K_M | 5.70 | 18.8 GB | 24 GB |
| Q4_K_M | 4.89 | **16.2 GB** | 24 GB |
| Q3_K_M | 4.00 | 13.2 GB | 16 GB |
| Q2_K | 3.16 | 10.4 GB | 12 GB |
| As measured: XQ mixed (~4.0 bpw routed experts, 13 of 24 layers keep offi | 4.00 | ≈13.2 GB | 16 GB |

Bits-per-weight for GGUF quants are llama.cpp's own figures ([llama.cpp quantize README](https://github.com/ggml-org/llama.cpp/blob/master/tools/quantize/README.md)). "As measured" rows apply the source's stated bits-per-weight to the parameter count; real files differ slightly because some layers stay at higher precision. Try other combinations in the [VRAM calculator](https://getrigbase.com/can-it-run/).

**Why it can still run on smaller GPUs:** it is a mixture-of-experts model, so only about 3.4B of its 26.4B parameters are used for each generated token. Runtimes such as llama.cpp (--n-cpu-moe), ExLlamaV3 or Project Maya can keep most expert weights in system RAM and only the shared layers and hot experts in VRAM. Then you need **VRAM + system RAM ≥ weights**, and speed is limited mainly by RAM bandwidth.

**Measured on setups with less GPU memory than the Q4_K_M / native weights** (so part of the model lived in system RAM or was streamed from disk):

- [AMD Radeon RX 9070 XT 16GB (16 GB) + 64 GB](https://getrigbase.com/builds/rx-9070-xt-16gb-inkling-2-mini-iq3xxs-vs-qwen38-27b-vulkan-windows/): **20.8 tok/s** (IQ3_XXS (InterpreticAI, 10.9 GiB))

## FAQ

### How much VRAM does Inkling 2 Mini need?

The weights are about 20 GB in its native format (Thinking Machines Lab model card: 26.4B total / 3.4B active looped MoE (middle layers run 3 times, ~5.4B effective active), text + image in, up to 1M context, Apache 2.0, released 20 Nov 2026; one ~20 GB NVFP4/MXFP8 checkpoint). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is RTX 3090 (24 GB) at 73.5 tok/s.

### What is the fastest measured hardware for Inkling 2 Mini?

2x RTX 3090: 136.6 tok/s generation (vLLM TP2 + Ampere overlay, NVLink, 1 stream, W4A16 INT4 + DSpark), from https://github.com/tonyd2wild/Inkling-2-Mini-W4A16-RTX3090.

### Which GPUs have measured results for Inkling 2 Mini?

2x RTX 3090, Apple M3 Ultra, Apple M5 Ultra, DGX Spark (GB10), RTX 3090, RTX 4090.

**Can it run on…** [2x RTX 3090](https://getrigbase.com/can-i-run/inkling-2-mini-on-2x-rtx-3090/) · [Apple M3 Ultra](https://getrigbase.com/can-i-run/inkling-2-mini-on-apple-m3-ultra/) · [Apple M5 Ultra](https://getrigbase.com/can-i-run/inkling-2-mini-on-apple-m5-ultra/) · [DGX Spark (GB10)](https://getrigbase.com/can-i-run/inkling-2-mini-on-dgx-spark-gb10/) · [RTX 3090](https://getrigbase.com/can-i-run/inkling-2-mini-on-rtx-3090/) · [RTX 4090](https://getrigbase.com/can-i-run/inkling-2-mini-on-rtx-4090/)

Related: [what GPU do you need for 2026 models](https://getrigbase.com/guides/what-gpu-for-qwen3-6-gemma-4-gpt-oss/) · [all builds](https://getrigbase.com/builds/)

**Search answer:** [How much VRAM do you need for Inkling 2 Mini?](https://getrigbase.com/vram/inkling-2-mini/) · [Will it fit my GPU?](https://getrigbase.com/can-it-run/)
