Qwen3.8 27B
on NVIDIA RTX 3090 · vLLM · 147,456 ctx
Sep 13, 2026
Use cases
tool-usemathlong-context
Summary
User reports Qwen3.8 27B at 38.39 t/s generation and 871.93 t/s prefill on an RTX 3090 24GB, averaged over 3 hours of constant running.
Setup is vLLM 0.27.1 with the Club 3090 patch, INT4 AutoRound weights, FP8 E4M3 KV cache, 147,456 token context, single sequence, and AOT compilation on WSL2.
At 10K prompt the run reached 42.3 t/s generation and 1000.26 t/s prefill; at 90K prompt it reached 34 t/s generation and 743.59 t/s prefill. The user notes llama.cpp with Q5 UD GGUF at 81,920 context plus MTP gave 25-30 t/s, and that 154K context was the practical ceiling.