llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX 5090 · Strata · 4,096 ctx

Tone: positiveNVIDIA hardware
Oct 24, 2026
Throughput
166.3 t/s gen · 3033.4 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
NVFP4 (NVFP4)
KV cache
int8
System RAM
128 GB
VRAM reported
32 GB

Use cases

agentictool-use

Summary

User compares Qwen3.8-Flash-Next NVFP4/GPTQ against Unsloth UD-Q6_K_XL on an RTX 5090 32 GB, with NVFP4 at 166.3 t/s generation and 3,033.4 t/s prompt processing at 4,096 tokens. Setup is Strata with int8 KV cache and a 131,072-token context ceiling; the speed figures are medians of three 256-token generations, and Q6's speed campaign had its control vector enabled. On a fresh 84-question MMLU-Pro subset NVFP4 scored 71/84 against Q6's 67/84 in 10m 57s versus 22m 18s, a difference the user calls not statistically established (McNemar p 0.219). Q6 reached 65.6 t/s at 4K and 2,094.4 t/s prefill, and an earlier mixed test had Q6 ahead on IFEval 9/10 to 7/10.