Qwen3.8 27B
on 2× NVIDIA RTX 3090 · vLLM · 262,144 ctx
Sep 30, 2026
Use cases
agentictool-uselong-context
Summary
User reports Qwen3.8-27B at 113.7 to 114.8 t/s decode and ~1,785 t/s prefill on 2x RTX 3090.
Setup is vLLM v0.29.0 with INT8 W8A16 compressed-tensors weights, fp8_e4m3 KV cache, 262,144 context, tensor parallel 2, DFlash2 W4A16 drafter with 7 speculative tokens, and a 64 GiB CPU offload tier.
User notes NVLink gives +12% decode and +18% prefill over PCIe P2P off, aggregate decode is 302.5 t/s at 4 concurrent, and a W8A8 build prefilled at 2,377 t/s at 46K.