Qwen3.8 125B (6B active) Flash-Next
on 4× Unknown GPU · llama.cpp · 131,072 ctx
Summary
User reports Qwen3.8-Flash-Next at 25.1 t/s on 4 GPUs at 128k context without speculative decoding.
Setup is llama.cpp with -sm layer across 4 GPUs. Enabling ngram-mod alone drops to 23.3 t/s (−7.1%), draft-mtp alone drops to 22.5 t/s (−10.3%), and MTP plus ngram drops to 20.6 t/s (−17.9%).
MTP acceptance is high at 82.9% but mean accepted length is only 2.8 tokens, so the draft head's expensive MoE forward per candidate is not amortized. ngram-mod has worse acceptance at 23.9% but longer drafts at 5.0 tokens; both spec implementations hurt at 128k.