More
Submit build

Builds / RTX 3090

RTX 3090: Inkling 2 Mini W4A16 INT4 on vLLM TP1 + Ampere overlay, 73.5 tok/s median (Tech2wild)

Last updated:

Source: github.com ↗More source: github.com ↗More source: huggingface.co ↗📏 How measured: reported by the original source; RigBase re-checked the numbers at that source
GPU(s)
RTX 3090
Amazon · eBay (used) · Newegg
VRAM / unified memory (GB)
24
CPU
AMD Ryzen 9 5900X
Amazon · eBay (used) · Newegg
Motherboard
not stated
RAM
32 GB
Amazon · eBay (used) · Newegg
PSU
not stated
Other
RTX 3090 (Ampere), one card of the same NVLink box
Reported total cost
not stated
Power
240 W GPU power limit (stock 350 W)

Measured speed (generation tokens/sec)

ModelQuantEnginetok/sPrompt processing (tok/s)Details
Inkling 2 MiniW4A16 INT4vLLM TP1 + Ampere overlay, 1 stream73.5not stated⚡ spec-decode (off)context: 8,192 default, 32,768 max (33,392-token BF16 KV pool at gpu-memory-utilization 0.95)📏 Median decode with no speculative decoding, greedy, one request at a time, as published in the quality table of Tech2wild's README (the lane table says ~74 tok/s). Tech2wild's calibration-free INT4 re-quantization of the routed experts (Tech2wild/Inkling-2-Mini-W4A16, 16.47 GB + 0.64 GB DSpark drafter); the stock image needs Tech2wild's Ampere overlay. 14.77 GiB of weights on one 24 GB card, so context is short. 18/19 fixed prompts.

Numbers by Tech2wild (tonyd2wild); Thinking Machines Lab (Inkling 2 Mini, DSpark drafter); vLLM project (vllm/vllm-openai:inkling-2-mini image, Humming expert kernels, Inkling 2 Mini recipe); compressed-tensors pack-quantized layout; Tech2wild/Inkling-2-Mini-W4A16 weights on Hugging Face. RigBase read the GitHub README on 2026-11-22 and checked each figure against the text. RigBase did not re-run it. Speculative decoding (MTP/DFlash/DSpark) inflates predictable text; these rows prefer prose/code over counting peaks. Recipe license: MIT.

Source: https://github.com/tonyd2wild/Inkling-2-Mini-W4A16-RTX3090
https://github.com/tonyd2wild/Inkling-2-Mini-W4A16-RTX3090/blob/main/README.md
https://huggingface.co/Tech2wild/Inkling-2-Mini-W4A16

Blank fields = not stated in the source. We never estimate build data; GPU street prices above are separately sourced and dated.

Embed this result

Copy into a README or blog. Links back to this build. See badge docs (CC BY 4.0 attribution). The SVG does not invent speeds — numbers stay on the build page.

Markdown

[![Measured on RigBase](https://getrigbase.com/badge/tony-rtx-3090-inkling-2-mini-w4a16-rtx3090.svg)](https://getrigbase.com/builds/tony-rtx-3090-inkling-2-mini-w4a16-rtx3090/)

HTML

<a href="https://getrigbase.com/builds/tony-rtx-3090-inkling-2-mini-w4a16-rtx3090/" rel="noopener"><img src="https://getrigbase.com/badge/tony-rtx-3090-inkling-2-mini-w4a16-rtx3090.svg" alt="Measured on RigBase" height="20"/></a>

Preview: Measured on RigBase · How to use

Rent before you buy: try this model on a cloud GPU for a few dollars first: Runpod (referral link; sign up with Google to get the new-user bonus)

Quick answers: Can RTX 3090 run Inkling 2 Mini?

Speculative-decoding rows on this page are not like-for-like with plain decode. See how much MTP and DFlash really speed things up (measured pairs).