Qwen3.8 27B Swift
4× NVIDIA RTX 3060 Ti 8GB · vLLM · 150,000 ctx
- generation:
- 120.0 tokens/s
- quant:
- W4A16 (safetensors)
- kv:
- bf16
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports 120 t/s on Qwen3.8-27B (Swift fine-tune) with a 150k context window on four RTX 3060 Ti 8GB cards. Setup is vLLM with W4A16 weights, bf16 KV cache, tensor parallel across 4 GPUs, and MTP speculative decoding; cards are power-limited to 110W. Quantizing the KV cache to kv8 extends context to the full 262k but drops speed to around 70 t/s. User also reports running 2 concurrent agents with little slowdown, and previously got about 70 t/s on a Swift-Qwen3.8-27B EXL3 4.00bpw model with MTP and 196k context under ExLlamaV3.