llamaperf

Qwen3.8 125B (6B active) Swift-1.5

on 2× NVIDIA DGX Spark · veloGB10

Tone: positiveNVIDIA hardware
Oct 5, 2026
Throughput
110.2 t/s gen
Quant
EXL3 4.05 (EXL3)

Use cases

codingagentictool-use

Summary

User reports Swift 1.5 (a fine-tune of Qwen3.8-Flash-Next) at ~110.2 tok/s single-stream decode on two DGX Sparks with veloGB10, TP=2. Setup is veloGB10 with an EXL3 4.05 pack (SharkWipf) and MTP speculative decoding; the official doth4580 pack reached 110.6 tok/s and a Huihui abliterated pack 113.6 tok/s on the same hardware. A header-only rewrite of the n-gram shard layout was needed to load community EXL3 packs. User compares against an older vLLM NVFP4 setup at ~52 tok/s (September, not same-day) and reports Swift 1.5 at xhigh effort passing 11/12 hard trap tasks versus 7/12 at medium, at about 2x output tokens and 1.8x wall time. Aggregate throughput at 2-3 concurrent requests is 91-98 tok/s, rising to 157 at 8 and 181 at 16.