Qwen3.8 27B
on AMD Radeon AI PRO R9700 32GB · vLLM · 262,144 ctx
Oct 4, 2026
Use cases
codingagenticlong-context
Summary
User reports Qwen3.8 27B at 179.7 tok/s single-stream decode on a single AMD Radeon AI PRO R9700 32GB.
Setup is stock vLLM with a plugin, 3-bit rotated GPTQ weights (W3A4) with FP8 KV cache, 262,144-token context, speculative decoding, and a 569,878-token prefix cache.
Aggregate throughput is 488.5 tok/s at 8 concurrent requests. Accuracy: GSM8K 95.22, HumanEval 94.51, MMLU-Pro 60.57. The 3-bit weights are a 9.55 GB add-on; MXFP4 remains available at 156.1 tok/s single-stream.