NVIDIA RTX Pro 6000 Blackwell reports

generation:

110.0 tokens/s

quant:

NVFP4 (safetensors)

mtp (multi-token prediction):

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next at roughly 110 t/s, ranging from 76 to 125 t/s, on a single RTX Pro 6000 Blackwell with 96 GB, at 165,000 to 170,000 tokens of context. Setup is vLLM with the NVFP4 quant and INT4 quantized PLE n-grams, MTP speculative decoding, prefix caching, and PLE CPU offload. The user praises quality for coding and agentic tasks.

generation:

109.1 tokens/s

prompt processing (prefill):

1955.0 tokens/s

quant:

UD-IQ4_XS (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User benchmarks Qwen3.8 Flash Next across VRAM tiers from 8 GB to 96 GB with llama.cpp b10666. CPU-only decode reaches 8.34 tok/s and full 96 GB decode reaches 109.07 tok/s. At 245K context, decode ranges from 14.89 tok/s on 24 GB to 21.61 tok/s on 96 GB. The model activates 6B params per token (MoE). RAM-resident loading gives 1.87x prefill versus mmap. Non-unified KV reaches 92.0 tok/s total at concurrency 16. A PLE table on CUDA causes a 55.6x decode slowdown, from 108.5 to 1.95 tok/s.

quant:

FP8

User benchmarks Qwen3.8-27B FP8 on one RTX PRO 6000 in Paddock, an open-source inference engine in Rust/C++ with its own CUDA kernels, reaching 1062 tok/s at 32 clients against 958 for vLLM and 844 for SGLang. Paddock is faster than vLLM in 13/13 cells, by 1.02x to 1.19x, and faster than llama.cpp Q8_0 in 13/13, by 1.5x to 37x.

generation:

40.0 tokens/s

prompt processing (prefill):

2000.0 tokens/s

quant:

IQ4 (GGUF)

kv:

Q8

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports prefill around 2000 t/s and decode around 40 t/s for a single request. With 2-4 parallel requests, prefill drops to 500-1000 t/s and decode to 10-20 t/s. User asks for a better setup for 2-4 slots with q4 quants or better and q8 KV cache or better, and mentions trying vLLM recipes before.

User reports Mimo 2.5 stays fast at large context on dual RTX Pro 6000, using 5-to-1 sliding-window attention similar to Gemma 3. User also mentions Step 3.7 Flash with 3-to-1 sliding-window attention at ~40 t/s at 178k context. User notes MiniMax M3 and DeepSeek V4 have kernel issues on Blackwell consumer GPUs.

User reports DeepSeek V4 Flash on 2x RTX PRO 6000 with vLLM reaches Sonnet quality and faster wall-clock time than Sonnet or Opus via API. User mentions Qwen 3.6 models as anchors.

generation:

57.4 tokens/s

quant:

Q6_K (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User benchmarks Tencent's Hy3, a 295B MoE with 21B active, at 57.4 tok/s generation with the Q6_K GGUF on 8x RTX PRO 6000 Blackwell. Setup is llama.cpp llama-bench with layer split and F16 KV cache, with the MTP/NextN block excluded. On the same cards Q4_K_M runs at 67.3 tok/s, Q3_K_L at 63.3 tok/s and IQ2_M at 78.7 tok/s. The user's Nemotron-Labs-Audex-30B-A3B quants run at 287 (Q8_0), 334 (Q5_K_M), 345 (Q4_K_M) and 329 (MXFP4_MOE) tok/s on 2x RTX PRO 6000 Max-Q.

generation:

67.0 tokens/s

quant:

AD-Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User compares AD-Q4_K_M, AD-Q5_K_M, AD-Q6_K, and Q8_0 quants of Qwen 3.8 27B on an RTX PRO 6000, reporting top-1 accuracy and KLD against BF16. Generation speeds are 67, 57, 49, and 50 tok/s respectively. The user recommends AD-Q6_K as the safest pick.

User benchmarks Qwen3.6 27B BF16 and Qwen3.6 35B BF16. For the 35B, best generation throughput is 3500 t/s at 128 concurrency with MTP off, and prompt throughput is 30000 t/s. The 27B was also tested with MTP on and off.

User reports FastDMS, a DMS KV-cache compression implementation, decoding 1.5-2x faster than vLLM BF16/FP8 with 5-8x less KV memory. Benchmarks cover Llama-3.2-1B and Qwen3-8B DMS checkpoints, with KLD and token match comparable or better than vLLM's FP8/TurboQuant. Training took about 20 min on an RTX Pro 6000 Blackwell.

User compares Qwen3.6-27B with and without thinking against Coder-Next. Qwen3.6-27B with thinking disabled is the most consistent at a 95.8% ship rate, and Qwen3.6-27B and Coder-Next are statistically tied overall. Qwen3.6-35B-A3B performed poorly and was dropped.