llamaperf

Qwen3.8 27B

on NVIDIA RTX 3090 · vLLM · 131,000 ctx

NVIDIA hardware
Sep 7, 2026
Throughput
218.3 t/s gen
Quant
INT4
VRAM reported
24 GB

Summary

User reports 120.1 t/s decode on narrative text and 218.3 t/s on code with vLLM v0.26.1rc1 and AutoRound INT4 (group 128) on 2x RTX 3090 over PCIe Gen4 x16/x16 with no NVLink and patched P2P. Prefill reaches 1342 tok/s at 10k and 628 tok/s at 90k. Speculative decoding uses 7 draft tokens with an acceptance length of 3.35 and 47.8% acceptance. Peak VRAM is 22.3 GB per card with zero leak, and the context ceiling is 131k because the DFlash2 drafter takes about 13.5 GB. Power is capped at 220/250 W, and the user used Kimi K3 for vLLM fixes.