llamaperf

Qwen3.8 27B

on 4× NVIDIA RTX 3090 · SGLang · 512 ctx

Tone: mixedNVIDIA hardware
Oct 8, 2026
Throughput
3843.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
W8A8 (safetensors)
KV cache
bf16
System RAM
94 GB
VRAM reported
24 GB

Use cases

codingagenticlong-context

Summary

User reports Qwen3.8-27B W8A8 on 4x RTX 3090 capped at 220 W with 3,843 t/s prefill at 512 prompt tokens (single request); decode is given only per task, 146.6 t/s narrative and 265.5 t/s code, with no overall figure. Setup is SGLang v0.5.19 on 4x RTX 3090 with TP=4, W8A8 INT8 weights, bf16 KV cache, DFlash2 speculative decoding with 8 draft tokens, HiCache ratio 1.5 and a 262144 context window. The c=1 prefill sweep ran 3,843 t/s at 512 tokens up to a 5,058 t/s peak at 4,096, and about 4,000-4,300 t/s at 10K and 3,029 t/s at 90K. All four GPUs were capped to 220 W of the 350 W default (62%), so every figure is power-limited. A concurrency sweep reached 1,811 t/s aggregate code decode at c=16 (113.2 t/s per request).