Qwen3.8 125B (6B active) Flash-Next
on NVIDIA DGX Spark · vLLM · 500,000 ctx
Sep 28, 2026
Use cases
agentictool-uselong-contextcoding
Summary
User reports Qwen3.8-Flash-Next at ~34 tok/s single-stream decode on a single NVIDIA DGX Spark (GB10, 128 GB unified pool).
Setup is vLLM v0.30.0 with NVIDIA's NVFP4 checkpoint, hybrid fp8 side layers, MTP=2 speculative decoding, prefix caching, deterministic top-k, and YaRN 500k context.
Prefill is ~2,700-4,000 tok/s (8k-32k prompts) with a ~505-520k-token KV pool. The repo patches vLLM to serve the 48 GiB n-gram PLE table from NVMe via mmap, fixing prefix caching and non-deterministic top-k on GB10. A separate concurrency sweep on the same hardware reports 266.8 aggregate tok/s at 48 concurrent streams.