Qwen3.8 27B
on NVIDIA RTX 3090 · sglang-exl3 · 208,858 ctx
Use cases
visionlong-contextcoding
Summary
User reports Qwen3.8-27B at 129.0 t/s decode and 786 t/s prefill on an RTX 3090.
Setup is sglang-exl3 v0.7.0-ampere with exl3-3bpw weights, MTP speculation, vision, and a 208k context window.
Protocol acceptance records prose 96.2 and code 141.1 tok/s per stream, cold prefill 1,523 tok/s at 32k and 710 at 209k, and a 4096² image TTFT of 22.5 s. A companion Qwen3.6-35B-A3B recipe at 256k reached 309.6 t/s decode and 1,907 t/s prefill.