llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX Pro 6000 Blackwell · Strata · 256,000 ctx

Tone: positiveNVIDIA hardware
Oct 11, 2026
Throughput
3319.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
UD-Q4_K_XL (GGUF)
KV cache
INT8
VRAM reported
96 GB

Use cases

coding

Summary

User reports Qwen3.8-Flash-Next UD-Q4_K_XL on an RTX PRO 6000 with Strata, INT8 KV cache at 256K context, and MTP-4 speculative decoding. Single-request decode varies by task: 185.8 t/s prose, 320.4 t/s counting, 298.9 t/s coding, 289.1 t/s reasoning, with prefill at 3,319 tok/s for a 17.3K prompt. vLLM on the same GPU with NVFP4 quant reached 95.3-171.0 t/s decode and 4,084 tok/s prefill, while vLLM on DGX Spark TP2 reached 38.2-75.3 t/s decode and 2,825 tok/s prefill.