Qwen3.8 27B
on 2× NVIDIA RTX 5060 Ti 16GB · vLLM · 262,144 ctx
Summary
User reports Qwen 3.8 27B NVFP4 on dual RTX 5060 Ti 16GB (32GB total VRAM) at 132 t/s decode for structured output and 72 t/s for prose, with ~4,000 tok/s prefill.
Setup is stock vLLM 0.30.0 on headless Ubuntu with FP8 KV cache, full 262,144-token context, and MTP speculative decoding. GPUs are power limited to 150W with no throughput cost.
Video input is disabled; the user notes this runs without an RTX 5090.