Qwen3.8 125B (6B active) Flash-Next
on 2× NVIDIA RTX 4070 · 98,000 ctx
Summary
User reports Qwen3.8-Flash-Next at 69 t/s at 98k tokens on an RTX 4070 12GB and Tesla V100 SXM2 16GB.
Setup uses IQ2_XS quant with 32GB DDR4 RAM and a 3500MB/s SSD, with CPU offload.
Speeds scale from 37 t/s fresh to 69 t/s at 98k tokens. User also gets 45-50 t/s on a 27B Q4_K_M model and asks whether IQ3_XXS would fit.