llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX Pro 6000 Blackwell · 100,000 ctx

Tone: positiveNVIDIA hardware
Oct 19, 2026
Throughput
81-88 t/s gen · 12000.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
NVFP4 (NVFP4)
VRAM reported
96 GB

Summary

User reports Qwen3.8 Flash-Next NVFP4 at 81-88 tokens per second on a single session on one RTX Pro 6000, without MTP. Setup follows the primitive-ai NVFP4 documentation; the 51B n-gram table needs 100GB of spare system memory. Prompt processing is GPU compute bound at 12000 tokens per second even with context scaled to 100k. Throughput scales to 341 tokens per second with 6 concurrent clients. The user calls the model great and is still benchmarking with MTP.