Qwen3.8 27B
on 4× MI100 · vLLM
Summary
User reports Qwen3.8 27B at 972 t/s TG and 5,680 t/s PP on 4x MI100.
Setup is a custom INT8 vLLM fork with AITER and a GPTQ INT8 quant.
User claims near-reference quality via KLD diagnostics.
on 4× MI100 · vLLM