Qwen3.8 125B (6B active) Flash-Next
on 2× NVIDIA RTX 5070 Ti · capy-flash
Summary
User reports Qwen3.8-Flash-Next 125B-A6B at ~100 tok/s generation and ~3.4k tok/s prefill on a 15k prompt on 2x RTX 5070 Ti 16GB.
Setup is the capy-flash engine with NVFP4 weights, experts offloaded to RAM, tensor parallel across both GPUs, on a Ryzen 9 9900X with 96 GB DDR5.
User compares against Strata, which gets 78 tok/s generation and 2.7k tok/s prefill with the same NVFP4 weights; Strata's 3-bit IQ3_S build is still faster at generation. User also notes a tensor-parallel finding: writing partial sums in FP16 instead of BF16 restores single-GPU numerical parity at no real speed cost.