llamaperf

NVIDIA RTX Pro 5000 Blackwell 72GB

NVIDIA · 72GB · 6 reports

Is this your card?

Engines people use on it: llama.cpp 4 (reports) · vLLM 1 (reports)

Run models on your NVIDIA RTX Pro 5000 Blackwell 72GB? Add your numbers.

Your coding agent can run a speed test on your machine and send it here with a free account. Or paste a result you've already posted.

Reported performance on the NVIDIA RTX Pro 5000 Blackwell 72GB

These reports mix GPU counts and memory setups, so a model listed here may have needed extra cards or CPU offloading. The calculator estimates which models fit in 72 GB of VRAM. Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.

Speeds are as each source reported them. GPU count, offloading and concurrent requests change them, and prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing two reports.

Tone: positiveNVIDIA hardware
generation:
140.7 tokens/s
quant:
NVFP4
kv:
fp8
codingagentictool-uselong-context

User benchmarks Qwen3.8 27B at 140.7 tok/s single-stream decode in NVFP4 on an RTX PRO 5000 72GB, up from 83.6 tok/s in FP8. Setup is vLLM 0.29.0 with FP8 KV cache, MTP speculative decoding with 3 tokens, chunked prefill and prefix caching, at 262144 context. Aggregate throughput is 300.0 tok/s at 8 concurrent streams and 321.0 at 16. A 200K prompt gives 85.7 s TTFT and 94.3 tok/s decode in NVFP4 versus 130.8 s and 33.2 tok/s in FP8. Quality on 300 items was 255/300 NVFP4 against 254/300 FP8. In the wider 800-test agentic coding leaderboard the local FP8 run scored 675/800 at 31.8 tok/s.

Nov 12, 2026
Tone: positiveNVIDIA hardware
generation:
280.0 tokens/s
quant:
Q8

User reports Qwen3.8 Flash-Next at Q8 running at 280 t/s on an RTX Pro 5000 Blackwell 72GB, with an RTX 5080 16GB and 192GB of system RAM also in the machine. Setup is the Basalt backend; the user compares it with Strata at 107 t/s on the same model. The user says Basalt works best with Blackwell and recommends Strata for older or mismatched generational hardware.

Nov 6, 2026
NVIDIA hardware
generation:
198.0 tokens/s
prompt processing (prefill):
5772.2 tokens/s
quant:
Q8_0 (GGUF)
kv:
F16
flash attention:
on

User reports Qwen3.6-35B-A3B Q8_0 at 198.0 t/s generation and 5772.2 t/s prompt processing on one NVIDIA RTX PRO 5000 Blackwell 72 GB. Setup is llama.cpp CUDA b10941 with Q8_0, f16 K and V cache, flash attention on, batch 2048, microbatch 512, and the whole model on GPU0. The 198.0 t/s is TG128, the mean of five repetitions; the 5772.2 t/s is PP512, a separate test, not a combined request. The author reports generation reproducibility around 1% and prompt-processing drift up to about 10% across sessions.

Oct 13, 2026
NVIDIA hardware
generation:
33.2 tokens/s
prompt processing (prefill):
2145.5 tokens/s
quant:
Q8_0 (GGUF)
kv:
F16
flash attention:
on

User reports Gemma 4 31B at 33.2 t/s generation and 2145.5 t/s prompt processing on one NVIDIA RTX PRO 5000 Blackwell 72GB. Setup is llama.cpp CUDA b10941 with Q8_0, f16 K and V cache, flash attention on, batch 2048 and microbatch 512, whole model on a single card. The 33.2 t/s is TG128 and the 2145.5 t/s is PP512, each the mean of five repetitions; the author reports generation reproducibility around 1% and prompt-processing drift up to about 10% across sessions.

Oct 13, 2026
NVIDIA hardware
generation:
38.5 tokens/s
prompt processing (prefill):
2677.5 tokens/s
quant:
Q8_0 (GGUF)
kv:
F16
flash attention:
on

User reports Muse Glimmer 30B at 38.5 t/s generation and 2677.5 t/s prompt processing on one NVIDIA RTX PRO 5000 Blackwell 72 GB. Setup is llama.cpp CUDA b10941 with Q8_0, f16 K and V cache, flash attention on, batch 2048 and microbatch 512, whole model on GPU0. The 38.5 t/s is TG128 and the 2677.5 t/s is PP512, each the mean of five repetitions; the author reports generation reproducibility around 1% and prompt-processing drift up to about 10% across sessions.

Oct 13, 2026
NVIDIA hardware
generation:
38.6 tokens/s
prompt processing (prefill):
2275.9 tokens/s
quant:
Q8_0 (GGUF)
kv:
F16
flash attention:
on

User reports Qwen3.8-27B at 38.6 t/s generation and 2275.9 t/s prompt processing on one NVIDIA RTX PRO 5000 Blackwell 72GB. Setup is llama.cpp CUDA b10941 with Q8_0, f16 K and V cache, flash attention on, batch 2048 and microbatch 512, whole model on the single card. The 38.6 t/s is TG128, generated tokens in a separate test from the PP512 figure; each cell is the mean of five repetitions, and the author reports prompt-processing drift up to about 10% across sessions.

Oct 13, 2026

Get a weekly email of new NVIDIA RTX Pro 5000 Blackwell 72GB reports and newly released models that fit it.

Email me new reports

Reported configurations

Individual observations from this page, not expected speeds or a ranking. Select a model to open its report, with the source and offloading conditions. Reported context may be a limit rather than actual input length.

Model / hardwareQuant / engineReported contextGeneration
Qwen3.8 27B
NVIDIA RTX Pro 5000 Blackwell 72GB
NVFP4
vLLM
262,144140.7 tokens/s
Qwen3.8 125B (6B active) Flash-Next
NVIDIA RTX Pro 5000 Blackwell 72GB
Q8
Basalt
Not reported280.0 tokens/s
Qwen3.6 35B (3B active)
NVIDIA RTX Pro 5000 Blackwell 72GB
Q8_0
llama.cpp
Not reported198.0 tokens/s
Gemma 4 30.7B
NVIDIA RTX Pro 5000 Blackwell 72GB
Q8_0
llama.cpp
Not reported33.2 tokens/s
Muse 27.9B Glimmer 30B
NVIDIA RTX Pro 5000 Blackwell 72GB
Q8_0
llama.cpp
Not reported38.5 tokens/s
Qwen3.8 27B
NVIDIA RTX Pro 5000 Blackwell 72GB
Q8_0
llama.cpp
Not reported38.6 tokens/s