llamaperf

Qwen3.8 27B

on NVIDIA RTX 3090 · vLLM · 65,536 ctx

Tone: positiveNVIDIA hardware
Sep 27, 2026
Throughput
138.0 t/s gen
Quant
GPTQ-int4 (GPTQ)
KV cache
fp8
VRAM reported
24 GB

Use cases

codinglong-context

Summary

User reports Qwen3.8-27B at ~138 t/s single-request on an RTX 3090, up from ~124 t/s, using a custom vLLM 0.27.1 stack with DFlash2 speculative drafting. Setup is vLLM with GPTQ-int4 weights, fp8 KV cache, 64k context, and a W4A16 DFlash2 drafter; the 3090 is power limited to 250W. The same stack reaches 942 t/s at 64 concurrent requests. User notes DFlash2 is best for 1-4 concurrent users and MTP wins at 8+ concurrent; quality is unchanged with perplexity 8.09 and GSM8K 96.5%.