llamaperf

Qwen3.8 27B

on NVIDIA RTX 3090 · SGLang · 262,144 ctx

NVIDIA hardware
Oct 8, 2026
Throughput
904.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
3.00bpw (EXL3)
KV cache
fp8
VRAM reported
24 GB

Summary

User reports Qwen3.8-27B EXL3 3.00bpw on a bare RTX 3090 24 GB through SGLang with the sglang-exl3 plugin, with cold prefill at 904 tok/s. Setup uses fp8 KV cache, MTP speculative decoding on, thinking off, one request, at the full 262,144-token window; the 2.4 GB token embedding sits in pinned host memory. Decode is given per workload: 98.8 t/s on prose, 143.3 on code and 89.6 on prose with a 32k prompt. A Qwen3.6-35B-A3B EXL3 recipe on the same card reached 251.9 prose and 352.1 code. A prior AWQ-INT4 27B recipe with no draft is cited at 45.1 t/s prose.