llamaperf

Qwen3.8 27B Swift-1.5-HyperQwen

on 4× NVIDIA RTX 3080 20GB · vLLM · 32,768 ctx

Tone: positiveNVIDIA hardware
Oct 3, 2026
Throughput
143.0 t/s gen · 889.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
W4A16 (safetensors)
KV cache
int8_per_token_head
System RAM
128 GB
VRAM reported
20 GB

Summary

User reports Qwen3.8-27B (Swift-1.5-HyperQwen W4A16) at 143 t/s decode and 889 t/s prefill at 32K context on 4x RTX 3080 20GB (modded) with tensor parallelism 4. Setup is vLLM 0.29.0 with W4A16 weights, int8_per_token_head KV cache, DFlash2 speculative decoding with 7 spec tokens, max_model_len 262144, max_num_seqs 4, chunked prefill and prefix caching enabled. Single-task decode ranges from 143 t/s at 32K to 240 t/s at 2K context; two concurrent tasks drop to 17-115 t/s. DFlash2 draft acceptance is 64.9% with 5.54 tokens/step. User previously got 40-50 t/s without DFlash2 and notes MTP degraded speeds.