llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX Pro 6000 Blackwell · llama.cpp · 245,760 ctx

NVIDIA hardware
Sep 7, 2026
Throughput
109.1 t/s gen · 1955.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
UD-IQ4_XS (GGUF)
System RAM
96 GB
VRAM reported
96 GB

Summary

User benchmarks Qwen3.8 Flash Next across VRAM tiers from 8 GB to 96 GB with llama.cpp b10666. CPU-only decode reaches 8.34 tok/s and full 96 GB decode reaches 109.07 tok/s. At 245K context, decode ranges from 14.89 tok/s on 24 GB to 21.61 tok/s on 96 GB. The model activates 6B params per token (MoE). RAM-resident loading gives 1.87x prefill versus mmap. Non-unified KV reaches 92.0 tok/s total at concurrency 16. A PLE table on CUDA causes a 55.6x decode slowdown, from 108.5 to 1.95 tok/s.