Qwen3.8 125B (6B active) Flash-Next
on 6× NVIDIA V100 16GB · 1cat-vllm · 262,144 ctx
Summary
User reports Qwen3.8 Next running on a 6x V100 16GB setup with TP2 PP3, reaching up to 4,679 t/s prefill at 16K input and 43.38 t/s generation with MTP enabled.
Setup uses the 1cat-vllm engine with speculative decoding limited to speculative=1; KV cache is 8.78 GiB, 531,288 tokens, supporting 2.03x concurrency at 262,144 tokens per request.
MTP nearly doubles generation throughput versus base (22.59 t/s average without MTP). A 20-minute gpu-burn stress test peaked at 65°C and stabilized at 64°C with fans at 76%.