llamaperf

Qwen3.8 27B

on 2× NVIDIA RTX 3090 · vLLM · 262,144 ctx

Tone: positiveNVIDIA hardware
Sep 30, 2026
Throughput
114.8 t/s gen · 1785.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
INT8 W8A16 (compressed-tensors)
KV cache
fp8_e4m3
System RAM
128 GB
VRAM reported
24 GB

Use cases

agentictool-uselong-context

Summary

User reports Qwen3.8-27B at 113.7 to 114.8 t/s decode and ~1,785 t/s prefill on 2x RTX 3090. Setup is vLLM v0.29.0 with INT8 W8A16 compressed-tensors weights, fp8_e4m3 KV cache, 262,144 context, tensor parallel 2, DFlash2 W4A16 drafter with 7 speculative tokens, and a 64 GiB CPU offload tier. User notes NVLink gives +12% decode and +18% prefill over PCIe P2P off, aggregate decode is 302.5 t/s at 4 concurrent, and a W8A8 build prefilled at 2,377 t/s at 46K.