Models / Ornith 1.5 35B-A3B / VRAM
How much VRAM do you need for Ornith 1.5 35B-A3B?
Last updated:
Weight math from the model card, plus every measured RigBase result for Ornith 1.5 35B-A3B. Leave extra room for context (KV cache).
Weight size by quant
| Quant | Bits/weight | Weights only |
|---|---|---|
| Q8_0 | 8.50 | 38.2 GB |
| Q6_K | 6.56 | 29.5 GB |
| Q5_K_M | 5.70 | 25.6 GB |
| Q4_K_M | 4.89 | 22.0 GB |
| Q3_K_M | 4.00 | 18.0 GB |
| Q2_K | 3.16 | 14.2 GB |
Bits-per-weight for GGUF rows: llama.cpp quantize README. Parameter count: Hugging Face safetensors total: 35.95B; model card: activates ~3B parameters per. We advise staying at Q4 or above for chat quality — see Is Q4 good enough?.
Measured speeds for Ornith 1.5 35B-A3B
| Hardware | Quant | Engine | tok/s |
|---|---|---|---|
| RTX 5090 32GB · 32 GB | Q4_K_M, f16 KV | llama-server b10448, no MTP | 273.2 |
| Intel Arc Pro B70 32 GB · 32 GB | Q4_K_M | llama.cpp SYCL + neural.download fusion | 131.46 |
| AMD Radeon RX 9060 XT 16GB · 16 GB | quant not stated (model tag ornith-1.5:35b-a3b) | llama.cpp (manual) | 95.8 |
| Apple M6 (12-core GPU, unified) · 32 GB | GGUF Q4_K_M | LM Studio (llama.cpp) | 37.6 |
⚡ = speculative decoding (MTP, DFlash…). Full list: best home rig for Ornith 1.5 35B-A3B.
Related
All Ornith 1.5 35B-A3B builds VRAM + KV calculator By VRAM size GGUF vs AWQ vs FP8 vs NVFP4 How much system RAM?
FAQ
How much VRAM do you need for Ornith 1.5 35B-A3B?
Weights alone are about 22.0 GB at Q4_K_M (35.95B × 4.89 bits ÷ 8). Add memory for the KV cache (context) and the runtime. The smallest-memory measured result in RigBase is 16 GB (AMD Radeon RX 9060 XT 16GB) at 95.8 tok/s. Going below Q4 usually hurts quality for chat; see the Q4 guide.
What is the fastest measured speed for Ornith 1.5 35B-A3B?
RTX 5090 32GB: 273.2 tok/s plain decode (Q4_K_M, f16 KV, llama-server b10448, no MTP).
Does system RAM replace VRAM?
Only for overflow. When the whole model fits in GPU or unified memory, system RAM barely matters for decode speed. See the system-RAM guide.
Referral link: if you sign up with Google and add $10, Runpod gives you and RigBase bonus credit (terms). It never affects our data.