llamaperf

Qwen3.8 27B

on NVIDIA RTX 3090 · vLLM · 147,456 ctx

Tone: positiveNVIDIA hardware
Sep 13, 2026
Throughput
38.4 t/s gen · 871.9 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
INT4 AutoRound (AutoRound)
KV cache
fp8_e4m3
System RAM
48 GB
VRAM reported
24 GB

Use cases

tool-usemathlong-context

Summary

User reports Qwen3.8 27B at 38.39 t/s generation and 871.93 t/s prefill on an RTX 3090 24GB, averaged over 3 hours of constant running. Setup is vLLM 0.27.1 with the Club 3090 patch, INT4 AutoRound weights, FP8 E4M3 KV cache, 147,456 token context, single sequence, and AOT compilation on WSL2. At 10K prompt the run reached 42.3 t/s generation and 1000.26 t/s prefill; at 90K prompt it reached 34 t/s generation and 743.59 t/s prefill. The user notes llama.cpp with Q5 UD GGUF at 81,920 context plus MTP gave 25-30 t/s, and that 154K context was the practical ceiling.