Qwen2.5 7B
on AMD RX 7900 XT · vLLM
Summary
User reports Qwen2.5-7B-Instruct-GPTQ-Int4 at 115 tok/s single-stream decode on a Radeon RX 7900 XT (gfx1100) under native Windows with AMD ROCm.
Setup is vLLM v0.19.1 with GPTQ Int4 weights, fp16 KV cache, native exllama W4A16 GEMM and hipGraph decode capture (FULL_DECODE_ONLY), gpu_memory_utilization=0.9, verified spill-free.
Aggregate throughput scales with concurrency: ~73 tok/s at batch 4, ~232 at batch 16, ~358 at batch 32. KVarN KV-cache quantization (K 4-bit / V 2-bit) gives 4.74x KV capacity (999,296 vs 210,784 tokens) at 74.7 tok/s, about 35% slower. A community-validated RX 7800 XT (gfx1101) reached 81.9 tok/s on the same model.