llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA DGX Spark · vLLM · 262,000 ctx

Tone: positiveNVIDIA hardware
Oct 3, 2026
Throughput
27.8 t/s gen · 1500-1730 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
NVFP4 (NVFP4)
KV cache
FP8
System RAM
128 GB

Summary

User compares vLLM and TensorFold running Qwen3.8-Flash-Next on a single DGX Spark (GB10, 128 GB unified memory). vLLM with an NVFP4 checkpoint, FP8 KV cache, and MTP with 3 draft tokens reaches 27.8 tok/s for a single request, while TensorFold with a 4-bit MLX checkpoint and int8 KV cache reaches 52.2 tok/s. At 5 concurrent requests, vLLM gives 78.5 tok/s and TensorFold 110.2 tok/s. Prompt processing is 1,500-1,730 tok/s for vLLM and 2,090-2,260 tok/s for TensorFold.