llamaperf

NVIDIA RTX Pro 5000 Blackwell 72GB reports

6 reports. These are individual setups, not a controlled benchmark. NVIDIA RTX Pro 5000 Blackwell 72GB speeds and models →

Speeds are as each source reported them. GPU count, offloading and concurrent requests change them, and prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing two reports.

Tone: positiveNVIDIA hardware
generation:
140.7 tokens/s
quant:
NVFP4
kv:
fp8
codingagentictool-uselong-context

User benchmarks Qwen3.8 27B at 140.7 tok/s single-stream decode in NVFP4 on an RTX PRO 5000 72GB, up from 83.6 tok/s in FP8. Setup is vLLM 0.29.0 with FP8 KV cache, MTP speculative decoding with 3 tokens, chunked prefill and prefix caching, at 262144 context. Aggregate throughput is 300.0 tok/s at 8 concurrent streams and 321.0 at 16. A 200K prompt gives 85.7 s TTFT and 94.3 tok/s decode in NVFP4 versus 130.8 s and 33.2 tok/s in FP8. Quality on 300 items was 255/300 NVFP4 against 254/300 FP8. In the wider 800-test agentic coding leaderboard the local FP8 run scored 675/800 at 31.8 tok/s.

Nov 12, 2026
Tone: positiveNVIDIA hardware
generation:
280.0 tokens/s
quant:
Q8

User reports Qwen3.8 Flash-Next at Q8 running at 280 t/s on an RTX Pro 5000 Blackwell 72GB, with an RTX 5080 16GB and 192GB of system RAM also in the machine. Setup is the Basalt backend; the user compares it with Strata at 107 t/s on the same model. The user says Basalt works best with Blackwell and recommends Strata for older or mismatched generational hardware.

Nov 6, 2026
NVIDIA hardware
generation:
198.0 tokens/s
prompt processing (prefill):
5772.2 tokens/s
quant:
Q8_0 (GGUF)
kv:
F16
flash attention:
on

User reports Qwen3.6-35B-A3B Q8_0 at 198.0 t/s generation and 5772.2 t/s prompt processing on one NVIDIA RTX PRO 5000 Blackwell 72 GB. Setup is llama.cpp CUDA b10941 with Q8_0, f16 K and V cache, flash attention on, batch 2048, microbatch 512, and the whole model on GPU0. The 198.0 t/s is TG128, the mean of five repetitions; the 5772.2 t/s is PP512, a separate test, not a combined request. The author reports generation reproducibility around 1% and prompt-processing drift up to about 10% across sessions.

Oct 13, 2026
NVIDIA hardware
generation:
33.2 tokens/s
prompt processing (prefill):
2145.5 tokens/s
quant:
Q8_0 (GGUF)
kv:
F16
flash attention:
on

User reports Gemma 4 31B at 33.2 t/s generation and 2145.5 t/s prompt processing on one NVIDIA RTX PRO 5000 Blackwell 72GB. Setup is llama.cpp CUDA b10941 with Q8_0, f16 K and V cache, flash attention on, batch 2048 and microbatch 512, whole model on a single card. The 33.2 t/s is TG128 and the 2145.5 t/s is PP512, each the mean of five repetitions; the author reports generation reproducibility around 1% and prompt-processing drift up to about 10% across sessions.

Oct 13, 2026
NVIDIA hardware
generation:
38.5 tokens/s
prompt processing (prefill):
2677.5 tokens/s
quant:
Q8_0 (GGUF)
kv:
F16
flash attention:
on

User reports Muse Glimmer 30B at 38.5 t/s generation and 2677.5 t/s prompt processing on one NVIDIA RTX PRO 5000 Blackwell 72 GB. Setup is llama.cpp CUDA b10941 with Q8_0, f16 K and V cache, flash attention on, batch 2048 and microbatch 512, whole model on GPU0. The 38.5 t/s is TG128 and the 2677.5 t/s is PP512, each the mean of five repetitions; the author reports generation reproducibility around 1% and prompt-processing drift up to about 10% across sessions.

Oct 13, 2026
NVIDIA hardware
generation:
38.6 tokens/s
prompt processing (prefill):
2275.9 tokens/s
quant:
Q8_0 (GGUF)
kv:
F16
flash attention:
on

User reports Qwen3.8-27B at 38.6 t/s generation and 2275.9 t/s prompt processing on one NVIDIA RTX PRO 5000 Blackwell 72GB. Setup is llama.cpp CUDA b10941 with Q8_0, f16 K and V cache, flash attention on, batch 2048 and microbatch 512, whole model on the single card. The 38.6 t/s is TG128, generated tokens in a separate test from the PP512 figure; each cell is the mean of five repetitions, and the author reports prompt-processing drift up to about 10% across sessions.

Oct 13, 2026