llama.cpp on the NVIDIA RTX 5070 Ti
llamaperf has 20 llama.cpp reports on the NVIDIA RTX 5070 Ti, none of them a plain single-card run with a model size and quant, so there is no median to quote. The reports are listed below with how each one ran.
20 llama.cpp reports on the NVIDIA RTX 5070 Ti
Newest first. A run on several cards, with speculative decoding, with part of the model in system RAM or with batched requests says so, since each describes a different setup.
- Qwen3.8 27B · UD-Q4_K_XL124.0 t/s(2 cards, speculative decoding)
- Gemma 4 31B · NVFP466.8 t/s(speculative decoding)
- Qwen3.6 35B · 3B active · Q4_K_M111.7 t/s(part in system RAM)
- Qwen3.8 125B · 6B active · Q428.0 t/s(2 cards, part in system RAM)
- Model not identified82.0 t/s(speculative decoding, part in system RAM)
- Qwen3.8 27B · UD-Q4_K_XL140.5 t/s(2 cards, speculative decoding)
- Qwen3.8 125B · 6B active · UD-IQ3_XXS20.4 t/s(speculative decoding, part in system RAM)
- Qwen3.8 27B · IQ3_S104.7 t/s(speculative decoding)
- Qwen3.8 27B · Q4_K_XL95.1 t/s(2 cards, speculative decoding)
- Qwen3.8 27B · UD-IQ4_XS50.0 t/s(speculative decoding)
- Agents-A1 35B · 3B active · APEX Compact97.0 t/s(speculative decoding)
- Qwen3.8 27B · UD-IQ4_XS50.0 t/s(part in system RAM)
- Qwen3.8 27B · Q3_K_XL72.8 t/s(speculative decoding)
- Qwen3.8 27B · UD-Q4_K_XL36.0 t/s(2 cards, speculative decoding)
- Qwen3.8 27B · Q4_K_XL22.3 t/s(2 cards)
- Qwen3.8 125B · 6B active · Q4_K_XL22.0 t/s(3 cards, part in system RAM)
- Qwen3.8 27B · IQ4_XS60.0 t/s(2 cards, speculative decoding)
- Qwen3.8 27B · Q4_K_XL115.0 t/s(2 cards, speculative decoding)
- Qwen3.8 125B · 6B active · IQ3_XSS26.0 t/s(part in system RAM)
- Qwen3.6 27B · Q4_K_M19.2 t/s(2 cards)
Frequently asked
How fast is llama.cpp on the NVIDIA RTX 5070 Ti?
llamaperf has 20 llama.cpp reports on the NVIDIA RTX 5070 Ti, none of them a plain single-card run with a model size and quant, so there is no median to quote. The reports are listed below with how each one ran.