Models / Ornith 1.5 35B-A3B

Best home rig for Ornith 1.5 35B-A3B

Current model35.95B total · 3B active1 measured result model info ↗

Will it fit my GPU? Best value GPU per euro →

Real, measured generation speeds for Ornith 1.5 35B-A3B on home hardware, fastest first. Different quants/engines aren't directly comparable.

Build	Quant	Engine	tok/s	Cost	Source
Single Intel Arc Pro B70 32 GB (neural.download lab)	Q4_K_M	llama.cpp SYCL + neural.download fusion patches; author marks this headline 'pending'	131.46		github.com ↗

Rent before you buy: try this model on a cloud GPU for a few dollars first: Runpod (referral link; sign up with Google to get the new-user bonus)

Memory math: how much VRAM / RAM does Ornith 1.5 35B-A3B need?

Ornith 1.5 35B-A3B has 35.95B parameters, about 3B active per token (source: Hugging Face safetensors total: 35.95B; model card: activates ~3B parameters per token). Weight size = parameters × bits-per-weight ÷ 8, before context (KV cache) and runtime overhead.

Format	Bits/weight	Weights	Smallest common memory size that holds the weights alone
Q8_0	8.50	38.2 GB	48 GB
Q6_K	6.56	29.5 GB	32 GB
Q5_K_M	5.70	25.6 GB	32 GB
Q4_K_M	4.89	22.0 GB	24 GB
Q3_K_M	4.00	18.0 GB	24 GB
Q2_K	3.16	14.2 GB	16 GB

Bits-per-weight for GGUF quants are llama.cpp's own figures (llama.cpp quantize README). "As measured" rows apply the source's stated bits-per-weight to the parameter count; real files differ slightly because some layers stay at higher precision. Try other combinations in the VRAM calculator.

Why it can still run on smaller GPUs: it is a mixture-of-experts model, so only about 3B of its 35.95B parameters are used for each generated token. Runtimes such as llama.cpp (--n-cpu-moe), ExLlamaV3 or Project Maya can keep most expert weights in system RAM and only the shared layers and hot experts in VRAM. Then you need VRAM + system RAM ≥ weights, and speed is limited mainly by RAM bandwidth.

FAQ

How much VRAM does Ornith 1.5 35B-A3B need?

The weights are about 22.0 GB at Q4_K_M (35.95B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is Intel Arc Pro B70 (32 GB) at 131.46 tok/s.

What is the fastest measured hardware for Ornith 1.5 35B-A3B?

Intel Arc Pro B70: 131.46 tok/s generation (llama.cpp SYCL + neural.download fusion patches; author marks this headline 'pending', Q4_K_M), from https://github.com/steveseguin/b70-optimization-lab.

Which GPUs have measured results for Ornith 1.5 35B-A3B?

Intel Arc Pro B70.

Can it run on… Intel Arc Pro B70