Single RTX 4090 (llama.cpp CUDA scoreboard)
| GPU(s) | RTX 4090 24GB Amazon · eBay (used) · Newegg |
|---|---|
| VRAM / unified memory (GB) | 24 |
| CPU | |
| Motherboard | |
| RAM | |
| PSU | |
| Other | llama-bench, flash attention on |
| Reported total cost | |
| Power |
Measured speed (generation tokens/sec)
| Model | Quant | Engine | tok/s |
|---|---|---|---|
| Llama 2 7B | Q4_0 | llama.cpp CUDA (FA) | 188.96 |
Current GPU street price (estimate, not the builder's cost)
New retail prices in Germany incl. VAT; used cards are often much cheaper. Prices change daily.
- 1× RTX 4090: ≈€3,955 each (≈$4,433), new, approx. price as of 2026-10-16, lowest listing on billiger.de (ASUS TUF Gaming RTX 4090 OG OC 24 GB)
New retail prices in Germany incl. VAT; used cards are often much cheaper. Prices change daily.
Source: https://github.com/ggml-org/llama.cpp/discussions/15013
Blank fields = not stated in the source. We never estimate build data; GPU street prices above are separately sourced and dated.
Rent before you buy: try this model on a cloud GPU for a few dollars first: RunPod