Qwen3.8 125B (6B active) Flash-Next
4× NVIDIA V100 32GB · vLLM · 512,000 ctx
- generation:
- 201.0 tokens/s
- quant:
- AWQ (AWQ)
- kv:
- FP8
coding
User reports Qwen3.8 Flash-Next at 201 tok/s on 4x V100 32GB PCIe cards with no NVLink, for a single request of English code. Setup is ReVolta 0.1.2, a vLLM fork, with an AWQ quant, 512K context, FP8 KV cache with CPU offload, TP4 plus DCP2, and DFlash2 speculative decoding drafting 8-token blocks from incoai's DFlash2 checkpoint. The user says this may be the first DFlash run on Volta (SM70); the table also lists a 163 figure beside the 201 tok/s row.