Qwen3.8 27B
on 4× 4x Intel Arc Pro B65 32GB · vLLM
Summary
User reports Qwen3.8-27B INT4 at 105.9 tok/s single-stream on 4x Intel Arc Pro B65 32GB with tensor parallelism 4 and MTP-2.
Setup is vLLM 0.31 with RedHat INT4 weights, 200 W cap per card, PCIe 4.0, on an AMD EPYC 7343 host with 499 GiB RAM.
At c32 the four-card setup reaches 712 tok/s aggregate; four one-card replicas reach 831 tok/s at c32. Prefill is 3,809 tok/s on an 8K prompt and 2,476 tok/s on 128K. Quality is +4.29% perplexity versus BF16.