llamaperf

Qwen3.8 125B (6B active) Flash-Next

on 2× NVIDIA RTX 5070 Ti · capy-flash

NVIDIA hardware
Oct 19, 2026
Throughput
178.0 t/s gen
Quant
NVFP4
System RAM
96 GB

Use cases

coding

Summary

User reports Qwen3.8-Flash-Next at 178 t/s on 2x RTX 5070 Ti with 96 GB DDR5, using their own capy-flash engine with NVFP4 weights. Setup keeps experts in VRAM with CPU fallback for misses; the same box gives Strata IQ3_S 195 t/s and Strata NVFP4 120 t/s. Prefill on capy-flash is 2.4k / 4.6k / 4.8k t/s at 4k / 16k / 64k. A replayed 53-turn coding session growing to 81k tokens took 177 s on capy-flash, 197 s on Strata IQ3_S and 268 s on Strata NVFP4. The post also lists many other engines' figures for Qwen3.8-27B and Flash-Next, and EU prices where 96 GB of DDR5 costs about as much as a GPU.