llamaperf

Qwen3.8 27B Swift-1.5

on AMD Radeon AI PRO R9700 32GB · vllm-sly-radiance · 65,536 ctx

AMD hardware
Oct 24, 2026
Throughput
154.4 t/s gen · 3927.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
MXFP4 (GPTQ)
KV cache
FP8
VRAM reported
32 GB

Summary

User reports Qwen3.8-27B (Swift-1.5 MXFP4 GPTQ) at 154.4 tok/s weighted single-stream decode and 3,927 tok/s prefill on one AMD Radeon AI PRO R9700 32GB. Setup is vllm-sly-radiance with MXFP4 weights, FP8 KV cache, 65,536 context, DFlash2 speculative decoding with a W4A16 drafter (k=7), 4,096 prefill chunk, prefix caching off. Prefill falls to 3,370 tok/s at 47k tokens; aggregate throughput is 480 tok/s at 8 concurrent requests and 512 tok/s at 16. Per-category decode ranges from 90.5 tok/s (prose) to 210.8 tok/s (json).