Qwen3.8 27B
on 2× NVIDIA RTX 3090 · vLLM · 262,000 ctx
Summary
User benchmarks Qwen3.8-27B on 4x RTX 3090 (24 GB each) with vLLM 0.20.1, reaching 587.4 tok/s at c32 with TP=2 on one NVLink pair versus 417.9 tok/s with TP=4.
Setup uses tensor parallelism across the cards, with a KV pool of 1.45M tokens on TP=4 and 503k tokens on TP=2, and a max context of 262k.
MTP speculative decoding is slower at all concurrency levels, and pipeline-parallel throughput stays flat with concurrency.