Builds / NVIDIA Jetson AGX Xavier (Volta SM72, unified)
NVIDIA Jetson AGX Xavier 32 GB (Volta SM72): LFM2.5 24B-A2B Q4_K_M at 36.9 tok/s and Qwen3.5 35B-A3B Q4_K_M at 24.6 on llama.cpp v1.0.0 CUDA
Last updated:
- GPU(s)
- NVIDIA Jetson AGX Xavier (Volta SM72, unified)
- VRAM / unified memory (GB)
- 32
- CPU
- Jetson AGX Xavier SoC (aarch64; llama-bench -t 8)
Amazon · eBay (used) · Newegg - Motherboard
- Jetson AGX Xavier module
Amazon · eBay (used) · Newegg - RAM
- 32 GB unified LPDDR4x (shared CPU/GPU)
- PSU
- not stated
- Other
- L4T R35.1.0, kernel 5.10.104-tegra, CUDA 11.4; llama.cpp v1.0.0 (11fe021), -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=72, FORCE_MMQ off (default); llama-bench -ngl 99 -t 8 -fa on, KV q4_0/q4_0, -r 3
- Reported total cost
- not stated
- Power
- not stated
Measured speed (generation tokens/sec)
| Model | Quant | Engine | tok/s | Prompt processing (tok/s) | Details |
|---|---|---|---|---|---|
| LFM2.5 24B-A2B | Q4_K_M (13.52 GiB), KV q4_0 | llama.cpp v1.0.0 11fe021 (CUDA 11.4, SM72) | 36.85 | 391 @ 512-token prompt est. TTFT @8k ≈ 21.0 s (calculated from measured prefill) | context: llama-bench tg128📏 tg128 36.85 ± 0.36 (v0.9.0: 36.32); system was loaded (a llama-server instance stayed resident), so absolute numbers may be slightly low. pp4096 369.95, pp16384 317.75. FP8 KV gave no gain |
| Qwen3.5 35B-A3B legacy | Q4_K_M (20.21 GiB) | llama.cpp v1.0.0 11fe021 (CUDA 11.4, SM72) | 24.6 | 228 @ 512-token prompt est. TTFT @8k ≈ 35.9 s (calculated from measured prefill) | context: llama-bench tg128📏 Same box and flags (-fa on), single summary figure given in the issue |
Prompt processing = prefill speed (how fast the prompt is read) as reported by the source, with the prompt length it was measured at; blank = the source does not report it. est. TTFT @8k is calculated from the measured prefill speed (8,192 ÷ prefill tok/s), shown only when prefill was measured on a prompt of 512+ tokens. It is not a measurement and ignores load time and caching. measured TTFT is the source's own time-to-first-token. Why prefill matters for coding assistants.
Owner's own llama-bench runs in a llama.cpp issue about Volta flash-attention configs (18 Nov 2026). The Jetson AGX Xavier is a 2018-era embedded board with 32 GB of shared memory, so MoE models with ~2-3B active parameters are what make it usable. The owner says forcing MMQ was slower on this Tegra part and q4_0 KV beat FP8 KV.
Blank fields = not stated in the source. We never estimate build data; GPU street prices above are separately sourced and dated.