Qwen3.8 27B Swift-1.5
on AMD Radeon AI PRO R9700 32GB · vllm-sly-radiance · 65,536 ctx
Summary
User reports Qwen3.8-27B (Swift-1.5 MXFP4 GPTQ) at 154.4 tok/s weighted single-stream decode and 3,927 tok/s prefill on one AMD Radeon AI PRO R9700 32GB.
Setup is vllm-sly-radiance with MXFP4 weights, FP8 KV cache, 65,536 context, DFlash2 speculative decoding with a W4A16 drafter (k=7), 4,096 prefill chunk, prefix caching off.
Prefill falls to 3,370 tok/s at 47k tokens; aggregate throughput is 480 tok/s at 8 concurrent requests and 512 tok/s at 16. Per-category decode ranges from 90.5 tok/s (prose) to 210.8 tok/s (json).