Qwen3.8 27B
on NVIDIA RTX 3090 · SGLang · 262,144 ctx
Oct 8, 2026
Summary
User reports Qwen3.8-27B EXL3 3.00bpw on a bare RTX 3090 24 GB through SGLang with the sglang-exl3 plugin, with cold prefill at 904 tok/s.
Setup uses fp8 KV cache, MTP speculative decoding on, thinking off, one request, at the full 262,144-token window; the 2.4 GB token embedding sits in pinned host memory.
Decode is given per workload: 98.8 t/s on prose, 143.3 on code and 89.6 on prose with a 32k prompt. A Qwen3.6-35B-A3B EXL3 recipe on the same card reached 251.9 prose and 352.1 code. A prior AWQ-INT4 27B recipe with no draft is cited at 45.1 t/s prose.