Qwen3.8 27B
on 4× NVIDIA V100 16GB · NInfer · 256,000 ctx
Summary
User reports Qwen3.8 27B at 353 t/s on four V100 16GB cards.
Setup is NInfer with NVFP4 weights, Q8 KV cache, 256k max context and MTP 4/0 speculation, tensor parallel across 4 GPUs.
The user previously ran 2 V100s at 181 t/s and notes the repo now supports tp4.