llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA DGX Spark · vLLM

Tone: positiveNVIDIA hardware
Sep 12, 2026
Throughput
43.9 t/s gen
Quant
NVFP4 (NVFP4)
KV cache
FP8
System RAM
128 GB

Use cases

tool-usevisionmultilingual

Summary

User reports Qwen3.8-Flash-Next (180B MoE, 7.31B active) at 43.9 t/s on a single DGX Spark, roughly four times the 11.2 t/s of Qwen3.8-27B dense on the same machine. Setup is vLLM with NVFP4 weights, FP8 KV cache, MTP depth 3, greedy sampling, thinking mode off, tensor parallel size 1, and 320 tokens written per test. The 123.53 GiB checkpoint fits in 121.7 GiB unified memory because the 51.2B-parameter PLE n-gram table (47.68 GiB) is read from NVMe per token while the 120.8B experts (56.25 GiB at NVFP4) stay resident. Tool calls, thinking mode, and image input work on the live server; video input was accepted but the test clip was faulty so that result is inconclusive.