Qwen3.8 125B (6B active) Flash-Next
on 2× NVIDIA RTX 5070 Ti · capy-flash
Summary
User reports Qwen3.8-Flash-Next at 178 tok/s decode and 4,828 tok/s prefill on a 64k prompt on two RTX 5070 Ti 16 GB cards.
Setup is the user's own capy-flash engine (a fork of NInfer) with NVFP4 weights, tensor parallel across both cards and a shared VRAM expert cache whose misses run on the CPU, on a Ryzen 9 9900X with 96 GB DDR5.
Figures are one request at a time, decode measured with LocalMaxxing reasoning-v1. The engine also runs Qwen3.8-27B.