Qwen3.8 125B (6B active) Flash-Next
on NVIDIA RTX 3070 · Strata · 1,000 ctx
Summary
User reports Qwen3.8-Flash-Next at about 40 t/s decode and 1000 t/s prefill on an RTX 3070.
Setup is Strata with an IQ3_S GGUF, int8 KV cache, 262144 max context, spec set to 4, and a q8 n-gram table instead of the bundled iq4_nl one. The model is 125B-A6B with experts partly offloaded, on a Ryzen 5 5600 with 64GB of RAM.
A vanilla config at the same context gave about 500 t/s prefill and 2-3 t/s less decode. The GPU unlock gave maybe 50-100 t/s extra prefill. The model runs from a SATA SSD, which the user thinks may hamper performance.