Qwen3.8 125B (6B active) Flash-Next
on NVIDIA DGX Spark · vLLM · 262,144 ctx
Use cases
codingagenticlong-context
Summary
User reports Qwen3.8 Flash at 60-70 (peak 74.1) tok/s peak single-stream decode on one DGX Spark (GB10), with 60 to 70 tok/s on normal requests and 212.2 tok/s aggregate across 8 streams.
Setup is vLLM with W4A16 AutoRound for the MoE experts, FP8 side layers and INT8 lm_head, full 262K context, using the model's own MTP draft head with a draft vocab cut from 248K to 65K ids.
Cold prefill reaches 4,016 tok/s at 16K, 2,426 tok/s at 64K and 2,213 tok/s at 128K, versus 1,171 / 1,071 / 1,065 tok/s for the original recipe. Cached coding prompts start replying in about 0.57 s. Quality scored 93.1% and 93.3% on a fixed 492-question suite over two seeds. The model takes about 71 GiB with a 16 GB KV pool; CJK drafting is slower due to the English/code draft vocab.