Qwen3.8 27B
on NVIDIA RTX 3090 · vLLM · 131,000 ctx
Sep 7, 2026
Summary
User reports 120.1 t/s decode on narrative text and 218.3 t/s on code with vLLM v0.26.1rc1 and AutoRound INT4 (group 128) on 2x RTX 3090 over PCIe Gen4 x16/x16 with no NVLink and patched P2P.
Prefill reaches 1342 tok/s at 10k and 628 tok/s at 90k. Speculative decoding uses 7 draft tokens with an acceptance length of 3.35 and 47.8% acceptance.
Peak VRAM is 22.3 GB per card with zero leak, and the context ceiling is 131k because the DFlash2 drafter takes about 13.5 GB. Power is capped at 220/250 W, and the user used Kimi K3 for vLLM fixes.