Qwen3.8 125B (6B active) Flash-Next
on 2× NVIDIA DGX Spark · vLLM · 512,000 ctx
Use cases
agenticlong-context
Summary
User reports Qwen3.8-Flash-Next at 30-50 tok/s single-stream decode on 2x NVIDIA DGX Spark linked by a direct ConnectX-7 cable.
Setup is vLLM with the RadixArk NVFP4 quant, TP=2 across both boxes, MTP speculative decoding with 3 tokens at about 40% acceptance, 512K context stretched from 262K with YaRN, and a 47.7 GiB n-gram embedding table mmap'd from NVMe instead of loaded into memory.
With about 9 concurrent agent sessions the engine reaches 181 tok/s aggregate, peaking at 195.