Qwen3.8 27B Swift-1.5-HyperQwen
on 4× NVIDIA RTX 3080 20GB · vLLM · 32,768 ctx
Summary
User reports Qwen3.8-27B (Swift-1.5-HyperQwen W4A16) at 143 t/s decode and 889 t/s prefill at 32K context on 4x RTX 3080 20GB (modded) with tensor parallelism 4.
Setup is vLLM 0.29.0 with W4A16 weights, int8_per_token_head KV cache, DFlash2 speculative decoding with 7 spec tokens, max_model_len 262144, max_num_seqs 4, chunked prefill and prefix caching enabled.
Single-task decode ranges from 143 t/s at 32K to 240 t/s at 2K context; two concurrent tasks drop to 17-115 t/s. DFlash2 draft acceptance is 64.9% with 5.54 tokens/step. User previously got 40-50 t/s without DFlash2 and notes MTP degraded speeds.