Qwen2.5 0.5B
on Adreno 740 · llama.cpp
Summary
User reports Qwen2.5-0.5B Q4_K_M at 69.3 t/s on CPU and 5.2 t/s on the OpenCL GPU path on an ASUS ROG Phone 7 (Snapdragon 8 Gen 2 / Adreno 740).
Setup is llama.cpp via a llama.rn fork with an experimental OpenCL backend, off by default; the lm_head K-quants are routed back to CPU to avoid an Adreno bug, and Flash Attention q8_0 is rejected on the A7X driver so attention falls back to CPU.
User notes the OpenCL path produced garbage tokens on this driver, and that Gemma 2 9B Q4_K_M was still about 2.1x slower on GPU. GPU is recommended only for 7B-9B+ models on drivers that do not miscompile K-quant GEMV.