Qwen3.8 27B
4× NVIDIA RTX 5070 Ti · vLLM · 65,536 ctx
- generation:
- 82.2 tokens/s
- prompt processing (prefill):
- 6488.4 tokens/s
- quant:
- FP8 (FP8)
User benchmarks Qwen3.8 27B FP8 on a quad RTX 5070 Ti 16GB node, reporting 82.21 t/s generation and 6488.44 t/s prompt processing at depth 0. Setup is vLLM 0.30 on Ubuntu 24.04 with CUDA 13.2, tensor-parallel-size 4, 65536 max model length, 0.85 GPU memory utilization, and no speculative decoding. GPUs are connected through a PLX 88096 PCIe switch with P2P enabled within the node. A second quad RTX 5060 Ti 16GB node reached 43.35 t/s generation and 3218.53 t/s prompt processing on the same model. Throughput scales roughly linearly with GPU tier, and the user notes results are stock with no overclock or undervolt.