Qwen3.8 125B (6B active) Flash-Next
on 4× NVIDIA V100 16GB · vLLM · 120,000 ctx
Summary
User reports Qwen3.8 Flash-Next 27B at over 200 tok/s generation on 4x V100 16GB GPUs.
Setup is vLLM with NVFP4 Nvidia checkpoints and DFlash speculative decoding, tensor parallel 4, with image support enabled and KV cache around 120k tokens.
Prefill is reported as 2.5-3.5k t/s. The user notes the server cost $5400 on eBay and used a custom repo to unpack NVFP4 into FP16 on the fly.