Qwen3.8 27B
on 2× NVIDIA RTX 3090 · vLLM · 262,144 ctx
Sep 22, 2026
Summary
User reports Qwen3.8 27B at over 70 t/s single-stream decode on 2x RTX 3090, with close to 200 t/s decode at 3 to 4 concurrency and over 10k t/s prefill.
Setup is stock vLLM v0.29 on Linux serving the RedHatAI INT4 quant with an FP8 KV cache, MTP speculative decoding with 3 tokens, tensor parallel size 2, and the full 262k context.
The user notes the setup is standard and that the RedHat quant is overlooked for Ampere hardware.