Qwen3.8 27B
on 2× NVIDIA RTX 3080 20GB · vLLM · 160,000 ctx
Use cases
agentictool-uselong-context
Summary
User reports Qwen3.8 27B at 57.7 t/s decode on two RTX 3080 20GB cards (40GB pool) at 160k context.
Setup is vLLM in Docker on Unraid with the AWQ-INT4 quant, tensor-parallel-size 2, max-num-seqs 1, max-num-batched-tokens 16384, and GPUs throttled to 250W. Prefill is roughly 2,500 t/s at 10k context with 4.23s TTFT.
Decode drops to about 51 t/s at 42k context and 43 t/s at 110k context; lifetime average across 611 agent requests is 47 t/s. User asks whether speeds can be improved.