Qwen3.8 125B (6B active) Flash-Next
on AMD RX 7900 XTX · TabbyAPI · 196,608 ctx
Use cases
codingagenticlong-context
Summary
User reports Qwen3.8-Flash-Next at 83 tok/s on code and 45-50 tok/s on prose on a single RX 7900 XTX 24GB with 60GB DDR5.
Setup is TabbyAPI with EXL3 3.05bpw, 128K context and 8-bit KV cache, 166 of 512 experts per layer on the GPU and 12 CPU threads, MTP drafting with 2 tokens.
A 28.7K-token code prompt prefills at 505 tok/s cold and decodes at 51-70 tok/s; without drafting the model runs 53 tok/s on code. The 4.05bpw variant reaches 68 tok/s on code. llama.cpp with IQ4_XS and --n-cpu-moe 36 is cited as a reference at 17.5 tok/s.