Models / Ling 3.0 Flash

Best home rig for Ling 3.0 Flash

Legacy model124B total · 5.1B active1 measured result model info ↗

Newer model available: Ling 3.1 Flash. Ling 3.0 Flash is a legacy model; its numbers stay here for reference. inclusionAI published Ling-3.1-flash on Hugging Face on 13 Oct 2026. source ↗

Real, measured generation speeds for Ling 3.0 Flash on home hardware, fastest first. Different quants/engines aren't directly comparable.

Build	Quant	Engine	tok/s	Prompt processing (tok/s)	Cost	Source
DGX Spark: Ling 3.0 Flash INT4 on SGLang, 83 tok/s (Mia's AI Lab recipe)	INT4 + DSpark	SGLang, 1 stream	83 ⚡	–		github.com ↗

Prompt processing = prefill speed reported by the source, @ prompt length. * est. TTFT @8k is calculated from measured prefill, not measured. Why it matters.

Rent before you buy: try this model on a cloud GPU for a few dollars first: Runpod (referral link; sign up with Google to get the new-user bonus)

Memory math: how much VRAM / RAM does Ling 3.0 Flash need?

Ling 3.0 Flash has 124B parameters, about 5.1B active per token (source: inclusionAI model card: 124B total, 5.1B activated (Hugging Face safetensors total 127.5B)). Weight size = parameters × bits-per-weight ÷ 8, before context (KV cache) and runtime overhead.

Format	Bits/weight	Weights	Smallest common memory size that holds the weights alone
Q8_0	8.50	131.8 GB	192 GB
Q6_K	6.56	101.7 GB	128 GB
Q5_K_M	5.70	88.4 GB	96 GB
Q4_K_M	4.89	75.9 GB	96 GB
Q3_K_M	4.00	61.9 GB	64 GB
Q2_K	3.16	49.0 GB	64 GB

Bits-per-weight for GGUF quants are llama.cpp's own figures (llama.cpp quantize README). "As measured" rows apply the source's stated bits-per-weight to the parameter count; real files differ slightly because some layers stay at higher precision. Try other combinations in the VRAM calculator.

Why it can still run on smaller GPUs: it is a mixture-of-experts model, so only about 5.1B of its 124B parameters are used for each generated token. Runtimes such as llama.cpp (--n-cpu-moe), ExLlamaV3 or Project Maya can keep most expert weights in system RAM and only the shared layers and hot experts in VRAM. Then you need VRAM + system RAM ≥ weights, and speed is limited mainly by RAM bandwidth.

FAQ

How much VRAM does Ling 3.0 Flash need?

The weights are about 75.9 GB at Q4_K_M (124B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is DGX Spark (GB10) (128 GB) at 83 tok/s.

What is the fastest measured hardware for Ling 3.0 Flash?

DGX Spark (GB10): 83 tok/s generation (SGLang, 1 stream, INT4 + DSpark), from https://github.com/MiaAI-Lab/Ling-3.0-Flash-SGLang-DSpark-DGX-Spark.