Qwen3.8 27B
on NVIDIA RTX 3090 · vLLM · 65,536 ctx
Sep 27, 2026
Use cases
codinglong-context
Summary
User reports Qwen3.8-27B at ~138 t/s single-request on an RTX 3090, up from ~124 t/s, using a custom vLLM 0.27.1 stack with DFlash2 speculative drafting.
Setup is vLLM with GPTQ-int4 weights, fp8 KV cache, 64k context, and a W4A16 DFlash2 drafter; the 3090 is power limited to 250W. The same stack reaches 942 t/s at 64 concurrent requests.
User notes DFlash2 is best for 1-4 concurrent users and MTP wins at 8+ concurrent; quality is unchanged with perplexity 8.09 and GSM8K 96.5%.