Qwen3.8 125B (6B active) Flash-Next
on 2× Huawei Atlas 300I Duo · vLLM · 262,144 ctx
Summary
User reports Qwen3.8 Flash-Next at 29.74 tok/s median (34.34 peak) single-request on two Huawei Atlas 300I Duo cards (four Ascend 310P3 ranks, 192 GB nameplate, ~172 GiB usable).
Setup is a forked vLLM and vLLM Ascend with W4A8 packed experts, MTP2 speculative decoding, ACL graphs, and a 262,144-token context limit; ECC disabled.
Four concurrent requests reach 60.88 tok/s aggregate; two concurrent 40K warm-prefix requests reach 30.76 tok/s aggregate; a 40K cold prompt takes about 120 seconds TTFT then 30-32 tok/s. A 106-case long-reasoning phase emitted 507,256 tokens in 2 h 52 m 46 s (48.93 tok/s aggregate, 12.50 tok/s median per request). GPQA Diamond scored 140/198 (70.71%) on the Ascend W4 service and the same on an RTX 6000 Pro with IQ4_XS GGUF in llama.cpp. GLM-5.3-Flash bring-up reaches 2.236 tok/s single-request and 4.476 tok/s aggregate at four requests.