llamaperf

NVIDIA RTX Pro 4000 Blackwell reports

7 reports. These are individual setups, not a controlled benchmark. NVIDIA RTX Pro 4000 Blackwell speeds and models →

Speeds are as each source reported them. GPU count, offloading and concurrent requests change them, and prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing two reports.

Qwen3.8 27B

2× NVIDIA RTX Pro 4000 Blackwell · NInfer · 39,843 ctx

NVIDIA hardware
generation:
111.1 tokens/s
prompt processing (prefill):
5634.0 tokens/s
quant:
QUASAR mixed NVFP4/FP8
codingmath

User benchmarks Qwen3.8-27B at 155.9 t/s decode on a ~40k-token prompt using two RTX PRO 4000 Blackwell 24GB cards in tensor parallel 2. Setup is NInfer v0.9.0 (ninfer-serve, CUDA) with a QUASAR mixed NVFP4/FP8 artifact, int8 KV cache, 262144-token context and DFlash2 speculative decoding with 7 draft tokens; GPUs are power-capped to 145 W each. A second long-context run gave 163.8 t/s decode and 5590 t/s prefill; prose decode was 111.1 t/s and short reasoning tasks averaged 204.7 t/s with 10/10 correct. A ~9k-token prompt had a median TTFT of 1.31 s. Timings are client-side, one request at a time.

Nov 9, 2026
Tone: positiveNVIDIA hardware
generation:
30-35 tokens/s
quant:
2.25 bpw

User reports a DeepSeek model at 2.25 bpw running at around 30-35 tokens/s on two RTX PRO 4000 Blackwell cards. The machine is a Xeon W-2245 workstation with 256GB of DDR4-2666 ECC RAM in 8 x 32GB sticks. No engine, context length or KV cache setting is stated. The speed is given only as a range, so no single figure is recorded.

Nov 4, 2026

Qwen3.8 27B

2× NVIDIA RTX Pro 4000 Blackwell · llama.cpp · 256,000 ctx

Tone: positiveNVIDIA hardware
generation:
68.0 tokens/s
quant:
Q8 (GGUF)
agentictool-usecodinglong-contextvisionsummarization

User reports a fully local voice and chat assistant running Qwen3.8-27B (dense, 27.3B, Q8) at roughly 68 tok/s decode and ~1.5K tok/s prefill on 2x RTX PRO 4000 Blackwell 24GB, with an RTX 5060 Ti 16GB handling display and helper models. Setup is a patched llama.cpp build for Blackwell with speculative decoding (MTP, ~62% draft acceptance) and a vision projector, serving 256K context across 2 concurrent slots. The machine is an AMD Threadripper 9960X with 128 GB DDR5 ECC on Ubuntu 26.04, drawing ~820 W across GPUs and CPU. The user built a custom Python proxy (~22 modules) in front of llama-server with tool-calling, context compaction, a document workspace, and six MCP servers (filesystem, SearXNG web search, ChromaDB RAG with bge-m3, reminders, documents, sandboxed code editor). A voice pipeline with wake word, faster-whisper STT and Piper TTS is currently paused. The user asks for recommendations and comments.

Oct 16, 2026
NVIDIA hardware
generation:
13.6 tokens/s
prompt processing (prefill):
338.4 tokens/s
quant:
MXFP4 (GGUF)

User reports DeepSeek-V4-Flash at 13.61 t/s generation and 338.41 t/s prompt eval on 2 x RTX PRO 4000 with 48 GiB VRAM and 160 GiB system RAM. Setup is llama.cpp with an MXFP4/Q8_0 GGUF (about 146 GiB) at 32768 context, using --fit on and --no-mmap. A llama-batched-bench sweep from 8K to 1M context shows prompt throughput falling from 448.57 t/s to 65.97 t/s and generation from 19.20 t/s to 6.56 t/s; without hc-ops the same sweep was slower (322.50 t/s prompt, 14.62 t/s generation at 8K). KLD against the base logits is 0.422277 mean for the Q2 quant and 0.115089 for the Q4 quant.

Oct 8, 2026
Tone: positiveNVIDIA hardware
generation:
71.4 tokens/s
quant:
AWQ (AWQ)
kv:
fp8_e5m2
codinglong-context

User benchmarks Qwen2.5-32B-Instruct-AWQ at 71.37 tok/s median decode on 2x RTX PRO 4000 Blackwell. Setup is SGLang 0.5.19 with AWQ weights, NGRAM speculative decoding (K=5), tensor parallelism 2 over PCIe Gen4, and FP8 KV cache. NGRAM speculative decoding gives a 1.28x median speedup over the 55.60 tok/s baseline, with high inter-prompt variance (std dev 19.18 tok/s). Per-domain decode ranges are 65-107 tok/s for JSON, 60-226 tok/s for code, and 58-82 tok/s for prose. FP8 KV cache alone measured 54.77 tok/s, a 1.5% regression.

Sep 26, 2026
Tone: positiveNVIDIA hardware
generation:
67.0 tokens/s
prompt processing (prefill):
785.0 tokens/s
kv:
Q8

User reports 67 t/s with MTP3 speculative decoding enabled, against 24.4 tok/s without MTP. Setup uses an INT8 KV cache with group-64. The figure comes from a 128K NIAH benchmark with 130,048 prompt tokens, where MTP acceptance was 100% on a deterministic answer. Roughly 727 MiB of VRAM was left.

Sep 7, 2026
Tone: positiveNVIDIA hardware
generation:
33.9 tokens/s

User reports Qwen3.8:27b at 33.91 t/s on 2x RTX PRO 4000s and an RTX PRO 2000, up from 12.2 t/s. Splitting the GPUs across VMs produced the gain. User also reports muse-glimmer:30B at 22.61 t/s on the same setup, up from 14.3 t/s.

Sep 7, 2026