Qwen3.8 27B
on NVIDIA RTX 5090 · vLLM
Summary
User reports Qwen3.8-27B at 86 tokens/sec median single-request generation on an RTX 5090, up from 51 tokens/sec with speculative decoding off.
Setup is vLLM on the 5090 (32 GB) in a tower with a Ryzen 9 9950X3D and 96 GB RAM; quant and context length are not stated.
The user notes speculative decoding left less memory for requests in flight, and that this was a generation benchmark rather than a 70% speedup on code reviews.