llamaperf

Qwen3.8 125B (6B active) Flash-Next

on 2× NVIDIA RTX 5070 Ti · capy-flash

NVIDIA hardware
Oct 24, 2026
Throughput
178.0 t/s gen
Quant
NVFP4 (NVFP4)
System RAM
96 GB

Summary

User reports Qwen3.8-Flash-Next at 178 tok/s decode and 4,828 tok/s prefill on a 64k prompt on two RTX 5070 Ti 16 GB cards. Setup is the user's own capy-flash engine (a fork of NInfer) with NVFP4 weights, tensor parallel across both cards and a shared VRAM expert cache whose misses run on the CPU, on a Ryzen 9 9900X with 96 GB DDR5. Figures are one request at a time, decode measured with LocalMaxxing reasoning-v1. The engine also runs Qwen3.8-27B.