More
Submit build

Builds / NVIDIA Jetson AGX Xavier (Volta SM72, unified)

NVIDIA Jetson AGX Xavier 32 GB (Volta SM72): LFM2.5 24B-A2B Q4_K_M at 36.9 tok/s and Qwen3.5 35B-A3B Q4_K_M at 24.6 on llama.cpp v1.0.0 CUDA

Last updated:

Source: github.com ↗📏 How measured: reported by the original source; RigBase re-checked the numbers at that source
GPU(s)
NVIDIA Jetson AGX Xavier (Volta SM72, unified)
VRAM / unified memory (GB)
32
CPU
Jetson AGX Xavier SoC (aarch64; llama-bench -t 8)
Amazon · eBay (used) · Newegg
Motherboard
Jetson AGX Xavier module
Amazon · eBay (used) · Newegg
RAM
32 GB unified LPDDR4x (shared CPU/GPU)
PSU
not stated
Other
L4T R35.1.0, kernel 5.10.104-tegra, CUDA 11.4; llama.cpp v1.0.0 (11fe021), -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=72, FORCE_MMQ off (default); llama-bench -ngl 99 -t 8 -fa on, KV q4_0/q4_0, -r 3
Reported total cost
not stated
Power
not stated

Measured speed (generation tokens/sec)

ModelQuantEnginetok/sPrompt processing (tok/s)Details
LFM2.5 24B-A2BQ4_K_M (13.52 GiB), KV q4_0llama.cpp v1.0.0 11fe021 (CUDA 11.4, SM72)36.85391 @ 512-token prompt
est. TTFT @8k ≈ 21.0 s (calculated from measured prefill)
context: llama-bench tg128📏 tg128 36.85 ± 0.36 (v0.9.0: 36.32); system was loaded (a llama-server instance stayed resident), so absolute numbers may be slightly low. pp4096 369.95, pp16384 317.75. FP8 KV gave no gain
Qwen3.5 35B-A3B legacyQ4_K_M (20.21 GiB)llama.cpp v1.0.0 11fe021 (CUDA 11.4, SM72)24.6228 @ 512-token prompt
est. TTFT @8k ≈ 35.9 s (calculated from measured prefill)
context: llama-bench tg128📏 Same box and flags (-fa on), single summary figure given in the issue

Prompt processing = prefill speed (how fast the prompt is read) as reported by the source, with the prompt length it was measured at; blank = the source does not report it. est. TTFT @8k is calculated from the measured prefill speed (8,192 ÷ prefill tok/s), shown only when prefill was measured on a prompt of 512+ tokens. It is not a measurement and ignores load time and caching. measured TTFT is the source's own time-to-first-token. Why prefill matters for coding assistants.

Owner's own llama-bench runs in a llama.cpp issue about Volta flash-attention configs (18 Nov 2026). The Jetson AGX Xavier is a 2018-era embedded board with 32 GB of shared memory, so MoE models with ~2-3B active parameters are what make it usable. The owner says forcing MMQ was slower on this Tegra part and q4_0 KV beat FP8 KV.

Source: https://github.com/ggml-org/llama.cpp/issues/32511

Blank fields = not stated in the source. We never estimate build data; GPU street prices above are separately sourced and dated.

Embed this result

Copy into a README or blog. Links back to this build. See badge docs (CC BY 4.0 attribution). The SVG does not invent speeds — numbers stay on the build page.

Markdown

[![Measured on RigBase](https://getrigbase.com/badge/jetson-agx-xavier-32gb-lfm25-24b-a2b-qwen35-35b-a3b-llamacpp-v1-cuda-sm72.svg)](https://getrigbase.com/builds/jetson-agx-xavier-32gb-lfm25-24b-a2b-qwen35-35b-a3b-llamacpp-v1-cuda-sm72/)

HTML

<a href="https://getrigbase.com/builds/jetson-agx-xavier-32gb-lfm25-24b-a2b-qwen35-35b-a3b-llamacpp-v1-cuda-sm72/" rel="noopener"><img src="https://getrigbase.com/badge/jetson-agx-xavier-32gb-lfm25-24b-a2b-qwen35-35b-a3b-llamacpp-v1-cuda-sm72.svg" alt="Measured on RigBase" height="20"/></a>

Preview: Measured on RigBase · How to use

Rent before you buy: try this model on a cloud GPU for a few dollars first: Runpod (referral link; sign up with Google to get the new-user bonus)