Qwen3.8 125B (6B active) Flash-Next
on 2× NVIDIA RTX 3090 · 262,144 ctx
Summary
User reports Qwen3.8-Flash-Next at 1,402 prompt tok/s and 135.2 output tok/s on 2x RTX 3090 with 128 GB system memory.
Setup uses the Intel AutoRound W4A16 target with FP8 PLE/ngram tensors. Prefill and decode use different test shapes and did not come from the same request.
A 262,016-token prompt with 128 output tokens prefilled at 1,275.6 tok/s and decoded at 54.5 tok/s. Warm decode over 4,096 output tokens ranged 127.1-134.0 tok/s. Maximum tested sequence was 262,144 tokens.