llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX 5090 · strata-nvfp4 · 262,144 ctx

Tone: positiveNVIDIA hardware
Oct 14, 2026
Throughput
116.0 t/s gen
Quant
NVFP4 (NVFP4)
KV cache
int8
System RAM
192 GB
VRAM reported
32 GB

Use cases

coding

Summary

User reports Qwen3.8-Flash-Next 125B-A6B at a median 116 t/s decode on an RTX 5090 32GB. Setup is the strata-nvfp4 fork with NVFP4 experts and Q8_0 dense weights, int8 KV cache, MTP draft head with spec 5, and 63 GiB of experts pinned in system RAM. Decode was median 116 t/s, mean 103, peak 171; prefill averaged 3,332 t/s with a 6,623 t/s peak. The 33m 45s run produced 124,443 output tokens over 87 steps with a 99.0% cache hit rate and no compactions.