llamaperf

Qwen3.8 27B

on NVIDIA RTX 3090 · ExLlamaV3 · 262,144 ctx

NVIDIA hardware
Sep 27, 2026
Throughput
162.9 t/s gen
Quant
EXL3-4.00bpw (EXL3)
KV cache
cq3
VRAM reported
24 GB

Summary

User reports Qwen3.8-27B at 162.9 tok/s decode on an RTX 3090 with native DFlash2 speculative decoding in ExLlamaV3. Setup is ExLlamaV3 (community fork, pin 355c6ee) with EXL3 4.00bpw target and DFlash2 draft, cq3 KV cache, at the full 262,144-token context. Acceptance length 5.66; autoregressive decode 42.8 tok/s and MTP head 116.3 tok/s for comparison. At 150k-token depth: 25.3 tok/s decode, 594 tok/s prefill, peak 23.13 GB.