Qwen3.8 125B (6B active) Flash-Next
on 4× NVIDIA V100 32GB · vLLM · 512,000 ctx
Use cases
coding
Summary
User reports Qwen3.8 Flash-Next at 201 tok/s on 4x V100 32GB PCIe cards with no NVLink, for a single request of English code.
Setup is ReVolta 0.1.2, a vLLM fork, with an AWQ quant, 512K context, FP8 KV cache with CPU offload, TP4 plus DCP2, and DFlash2 speculative decoding drafting 8-token blocks from incoai's DFlash2 checkpoint.
The user says this may be the first DFlash run on Volta (SM70); the table also lists a 163 figure beside the 201 tok/s row.