llamaperf

Qwen3.8 27B Swift-1.5

on 2× NVIDIA RTX 3080 20GB · vLLM · 8,192 ctx

Tone: positiveNVIDIA hardware
Oct 25, 2026
Throughput
2080.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
AWQ-INT4 (AWQ)
KV cache
FP8
VRAM reported
20 GB

Use cases

agenticcodinglong-context

Summary

User benchmarks Qwen3.8 27B (Swift-1.5, AWQ-INT4) on dual RTX 3080 20GB cards in tensor parallel, reporting 2080 t/s prefill at 8192 prompt tokens with gpu-unlock plus P2P versus 967.1 t/s stock, a 115% gain. Setup is vLLM 0.32.0 with FP8 KV cache, MTP speculative decoding with 3 tokens, and both cards limited to 240W in PCIe 3.0 x16 slots. Prefill gains range from 70% to 115% across prompts of 8192 to 261900 tokens. Decode is reported as inter-token latency step rates of about 15 to 16.6 steps/s, which the user says corresponds to roughly 45 tok/s at an MTP acceptance length of 2; decode changes are within noise except at 261k tokens.