Qwen3.8 125B (6B active) Flash-Next
on 2× NVIDIA RTX 5070 Ti · Strata
Summary
User benchmarks Qwen3.8-Flash-Next at 195 t/s decode on two RTX 5070 Ti cards with 96 GB of system RAM, using Strata with IQ3_S weights and experts offloaded to RAM.
Prefill on the same box is 2.2k t/s at 3.5k tokens, 3.8k at 15k and 5.2k at 55k.
On the same hardware capy-flash with NVFP4 and tensor parallel reaches 171-183 t/s decode, and Strata NVFP4 reaches 120 t/s. The user reports Strata IQ3_S beats NVFP4 engines on short decode at about twice the KL divergence (0.144 for IQ3_S vs 0.070 for NVFP4).