Qwen3.8 27B
on NVIDIA RTX 3090 · vLLM · 262,144 ctx
Sep 7, 2026
Summary
User reports Qwen3.8-27B at about 124 t/s greedy on a single RTX 3090.
Setup uses fp8 KV cache, int8 lm_head and embed_tokens, MTP-4 drafts, GPTQ-int4 lm_head and MTP module, a split-KV attention kernel, and KVarN 4/2-bit KV cache for 262k context.
Peak concurrent throughput is about 1000 t/s at 64 concurrent.