More
Submit build

Models / Inkling 2 Mini / VRAM

How much VRAM do you need for Inkling 2 Mini?

Last updated:

Weight math from the model card, plus every measured RigBase result for Inkling 2 Mini. Leave extra room for context (KV cache).

Quick answer: the native release weights are about 20 GB. only about 3.4B parameters are active per token (MoE). the smallest-memory build with a measured result here is AMD Radeon RX 9070 XT 16GB (16 GB) at 20.8 tok/s. Open the VRAM calculator →

Weight size by quant

QuantBits/weightWeights only
Native release format–≈20 GB
Q8_08.5028.1 GB
Q6_K6.5621.7 GB
Q5_K_M5.7018.8 GB
Q4_K_M4.8916.2 GB
Q3_K_M4.0013.2 GB
Q2_K3.1610.4 GB

Bits-per-weight for GGUF rows: llama.cpp quantize README. Parameter count: Thinking Machines Lab model card: 26.4B total / 3.4B active looped MoE (middle l. We advise staying at Q4 or above for chat quality — see Is Q4 good enough?.

Measured speeds for Inkling 2 Mini

HardwareQuantEnginetok/s
2x RTX 3090 · 48 GBW4A16 INT4 + DSparkvLLM TP2 + Ampere overlay, NVLink, 1 str136.6 ⚡
2x RTX 3090 · 48 GBW4A16 INT4 + DSparkvLLM TP2 + Ampere overlay, NVLink, 1 str119 ⚡
2x RTX 3090 · 48 GBW4A16 INT4 + DSparkvLLM TP2 + Ampere overlay, NVLink P2P of100.5 ⚡
2x RTX 3090 · 48 GBW4A16 INT4vLLM TP2 + Ampere overlay, 1 stream89.8 ⚡
Apple M5 Ultra 80-core GPU (unified, Mac Studio) · 96 GBoMLX oQ8e, Affine8 8-bit KVoMLX v0.8.1, Lightning MTP83.4 ⚡
Apple M5 Ultra 80-core GPU (unified, Mac Studio) · 96 GBoMLX oQ8e, Affine8 8-bit KVoMLX v0.8.1, Lightning MTP76.6 ⚡
Apple M5 Ultra 80-core GPU (unified, Mac Studio) · 96 GBoMLX oQ8e, Affine8 8-bit KVoMLX v0.8.1, Lightning MTP74.7 ⚡
RTX 3090 · 24 GBW4A16 INT4vLLM TP1 + Ampere overlay, 1 stream73.5 ⚡
RTX 4090 · 24 GBXQ mixed (~4.0 bpw routed experts, 13 of 24 layersXolotl (xolotl-inkling)59.7
RTX 4090 · 24 GBOfficial precision, exact conversion (NVFP4 + MXFPXolotl (xolotl-inkling)57
Apple M5 Ultra 80-core GPU (unified, Mac Studio) · 96 GBoMLX oQ8e, Affine8 8-bit KVoMLX v0.8.1, Lightning MTP56.2 ⚡
Apple M5 Ultra 80-core GPU (unified, Mac Studio) · 96 GBoMLX oQ8e, Affine8 8-bit KVoMLX v0.8.1, Lightning MTP55.4 ⚡

⚡ = speculative decoding (MTP, DFlash…). Full list: best home rig for Inkling 2 Mini.

Related

All Inkling 2 Mini builds VRAM + KV calculator By VRAM size GGUF vs AWQ vs FP8 vs NVFP4 How much system RAM?

FAQ

How much VRAM do you need for Inkling 2 Mini?

Weights alone are about 20 GB in the native release format. Add memory for the KV cache (context) and the runtime. The smallest-memory measured result in RigBase is 16 GB (AMD Radeon RX 9070 XT 16GB) at 20.8 tok/s. Going below Q4 usually hurts quality for chat; see the Q4 guide.

What is the fastest measured speed for Inkling 2 Mini?

RTX 4090: 59.7 tok/s plain decode (XQ mixed (~4.0 bpw routed experts, 13 of 24 layers keep official NVFP4; Q8_0 backbone; 13.93 GB), abliterated, Xolotl (xolotl-inkling)).

Does system RAM replace VRAM?

Only for overflow. When the whole model fits in GPU or unified memory, system RAM barely matters for decode speed. See the system-RAM guide.

Not enough VRAM? Rent before you buy. Run Inkling 2 Mini on a rented 24–96 GB cloud GPU by the hour and see the real speed before spending €1,000+. Try it on Runpod
Referral link: if you sign up with Google and add $10, Runpod gives you and RigBase bonus credit (terms). It never affects our data.