Qwen3.8 125B (6B active) Flash-Next
on NVIDIA DGX Spark · vLLM · 262,144 ctx
Oct 3, 2026
Summary
User reports Qwen3.8-Flash-Next at 43.9 tok/s median single-stream on one DGX Spark (GB10, 128 GB unified memory).
Setup is vLLM nightly 8a728663 with the official NVIDIA NVFP4 checkpoint, FP8 KV cache, MTP3 speculative decoding, 6 sequences, 4096-token prefill chunk, 262,144-token context, gpu-memory-utilization 0.80; the 47.68 GiB FP8 n-gram table stays on NVMe and is read on demand, giving a KV pool of 1,027,392 tokens.
Under load the same boot gives 33.4 tok/s per stream at x2 (43.9 aggregate), 26.4 at x4 (47.1), 21.7 at x6 (68.8). Cold prefill is 1,269 tok/s at 7K, 1,757 at 28K and 1,760 at 113K. The user also compares TP2 and TP4 lanes, where TP2 SPEED reaches 53.7 tok/s median single-stream and TP4 CONTEXT 40.5.