llamaperf

Qwen3.8 125B (6B active) Flash-Next

on 2× NVIDIA DGX Spark · vLLM · 512,000 ctx

Tone: positiveNVIDIA hardware
Aug 29, 2026
Throughput
30-50 t/s gen
Quant
NVFP4
System RAM
128 GB
VRAM reported
128 GB

Use cases

agenticlong-context

Summary

User reports Qwen3.8-Flash-Next at 30-50 tok/s single-stream decode on 2x NVIDIA DGX Spark linked by a direct ConnectX-7 cable. Setup is vLLM with the RadixArk NVFP4 quant, TP=2 across both boxes, MTP speculative decoding with 3 tokens at about 40% acceptance, 512K context stretched from 262K with YaRN, and a 47.7 GiB n-gram embedding table mmap'd from NVMe instead of loaded into memory. With about 9 concurrent agent sessions the engine reaches 181 tok/s aggregate, peaking at 195.