Qwen3.8 27B
on 2× NVIDIA RTX 3090 · vLLM · 262,144 ctx
Use cases
codingagentictool-use
Summary
User reports sustained 80-90 t/s decode at full 262K context.
Setup uses MTP speculative decoding and prefix caching, with a 96.8% prefix cache hit rate and mean TTFT of 6.5s.
Multi-session throughput reaches about 3200 tokens/min across concurrent chats, and the benchmark ceiling is 106.9 t/s.