Qwen3.8 125B (6B active) Swift-1.5
on 2× NVIDIA DGX Spark · veloGB10
Use cases
codingagentictool-use
Summary
User reports Swift 1.5 (a fine-tune of Qwen3.8-Flash-Next) at ~110.2 tok/s single-stream decode on two DGX Sparks with veloGB10, TP=2.
Setup is veloGB10 with an EXL3 4.05 pack (SharkWipf) and MTP speculative decoding; the official doth4580 pack reached 110.6 tok/s and a Huihui abliterated pack 113.6 tok/s on the same hardware. A header-only rewrite of the n-gram shard layout was needed to load community EXL3 packs.
User compares against an older vLLM NVFP4 setup at ~52 tok/s (September, not same-day) and reports Swift 1.5 at xhigh effort passing 11/12 hard trap tasks versus 7/12 at medium, at about 2x output tokens and 1.8x wall time. Aggregate throughput at 2-3 concurrent requests is 91-98 tok/s, rising to 157 at 8 and 181 at 16.