Models / Kolibri-1

Best home rig for Kolibri-1

Current model78.1B total · 3.46B active1 measured resultfastest plain decode: 58.6 tok/s model info ↗

Will it fit my GPU? Best value GPU per euro →

Real, measured generation speeds for Kolibri-1 on home hardware, fastest first. Different quants/engines aren't directly comparable.

Build	Quant	Engine	tok/s	Cost	Source
Strix Halo (Ryzen AI Max+ 395, 128 GB) running Kolibri-1 Q4_K_M	Q4_K_M (47.45 GB)	llama.cpp ROCm (Seraphiel102 fork 8798b65), -ngl 99, f16 KV	58.6		git.mannheim.ccc.de ↗

Rent before you buy: try this model on a cloud GPU for a few dollars first: Runpod (referral link; sign up with Google to get the new-user bonus)

Memory math: how much VRAM / RAM does Kolibri-1 need?

Kolibri-1 has 78.1B parameters, about 3.46B active per token (source: Aleph Alpha model card: 78,103,074,560 total, 3,457,573,120 active per token). Weight size = parameters × bits-per-weight ÷ 8, before context (KV cache) and runtime overhead.

Format	Bits/weight	Weights	Smallest common memory size that holds the weights alone
Q8_0	8.50	83.0 GB	96 GB
Q6_K	6.56	64.1 GB	96 GB
Q5_K_M	5.70	55.7 GB	64 GB
Q4_K_M	4.89	47.8 GB	48 GB
Q3_K_M	4.00	39.0 GB	48 GB
Q2_K	3.16	30.8 GB	32 GB

Bits-per-weight for GGUF quants are llama.cpp's own figures (llama.cpp quantize README). "As measured" rows apply the source's stated bits-per-weight to the parameter count; real files differ slightly because some layers stay at higher precision. Try other combinations in the VRAM calculator.

Why it can still run on smaller GPUs: it is a mixture-of-experts model, so only about 3.46B of its 78.1B parameters are used for each generated token. Runtimes such as llama.cpp (--n-cpu-moe), ExLlamaV3 or Project Maya can keep most expert weights in system RAM and only the shared layers and hot experts in VRAM. Then you need VRAM + system RAM ≥ weights, and speed is limited mainly by RAM bandwidth.

FAQ

How much VRAM does Kolibri-1 need?

The weights are about 47.8 GB at Q4_K_M (78.1B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is Strix Halo (Radeon 8060S) (128 GB) at 58.6 tok/s.

What is the fastest measured hardware for Kolibri-1?

Strix Halo (Radeon 8060S): 58.6 tok/s generation (llama.cpp ROCm (Seraphiel102 fork 8798b65), -ngl 99, f16 KV, Q4_K_M (47.45 GB)), from https://git.mannheim.ccc.de/C3MA/AI_Benchmarks/commits/branch/main.

Which GPUs have measured results for Kolibri-1?

Strix Halo (Radeon 8060S).

Can it run on… Strix Halo (Radeon 8060S)