llama.cpp on the NVIDIA RTX 4080
llamaperf has 6 llama.cpp reports on the NVIDIA RTX 4080, none of them a plain single-card run with a model size and quant, so there is no median to quote. The reports are listed below with how each one ran.
6 llama.cpp reports on the NVIDIA RTX 4080
Newest first. A run on several cards, with speculative decoding, with part of the model in system RAM or with batched requests says so, since each describes a different setup.
- Qwen3.8 27B · IQ3_XXS70.0 t/s(speculative decoding)
- Qwen3.8 27B · UD-IQ4_XS55.0 t/s(speculative decoding)
- Qwen3.8 125B · 6B active · Q4_K_M16.5 t/s(speculative decoding, part in system RAM)
- Qwen3.6 35B · 3B active · MXFP4_MOE45.0 t/s(speculative decoding, part in system RAM)
- Qwen3.8 27B · IQ4_XS19.0 t/s(speculative decoding, part in system RAM)
- Qwen3.8 125B · 6B active · Q4_K_M8.0 t/s(part in system RAM)
Frequently asked
How fast is llama.cpp on the NVIDIA RTX 4080?
llamaperf has 6 llama.cpp reports on the NVIDIA RTX 4080, none of them a plain single-card run with a model size and quant, so there is no median to quote. The reports are listed below with how each one ran.