llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX 5090 · Basalt

Tone: positiveNVIDIA hardware
Oct 18, 2026
Throughput
8582.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
NVFP4 (NVFP4)
System RAM
128 GB

Summary

User reports Qwen3.8 Flash-Next running natively in NVFP4 under Basalt on an RTX 5090 at 400W with 128 GB of DDR5 RAM, giving 304 tok/s on structured output and 183 tok/s on prose. Setup is Basalt with a .basalt pack that keeps stock NVIDIA experts and stores the dense layers as Q6_K; adding a peer RTX 5060 Ti raises the figures to 479 tok/s structured and 239 tok/s prose. Prefill reaches 8,582 tok/s on a 64K prompt, 7.6s to first token. Against Infernix on the same NVFP4 weights and box the user claims up to 1.3x on decode and 1.6x on prefill. Quality versus Q8_0 over 100 chunks: PPL 3.308 (Q8_0 3.286), KLD 0.068, 91.5% same top-1.