Local model performance on your hardware

generation:

21.2 tokens/s

prompt processing (prefill):

568.5 tokens/s

quant:

Q4_K_M (GGUF)

User reports 21.16 t/s generation and 568.54 t/s prompt processing. Source evidence (verbatim): Source: https://blog.majid.info/llama-bench/ai-x1-pro.txt # AMD Ryzen AI 9 HX 370 / Radeon 890M, 96GB system RAM: original Gemma4 E4B Q4_K_M llama-bench log First-hand author log linked from https://blog.majid.info/llama-bench/ (April 13, 2026 article). Measured hardware: AMD Ryzen AI 9 HX 370 / Radeon 890M, 96GB system RAM. Gemma4 E4B Q4_K_M GGUF, llama.cpp 69c28f1 (1), backend ROCm. pp512 568.54 ±1.96 tokens/s; tg128 21.16 ±0.03 tokens/s. gfx1150; ngl99; runtime reports 47072 MiB device VRAM visibility; integrated unified memory, not dedicated VRAM or measured resident use. Command names --hf-repo unsloth/gemma-4-E4B-it-GGUF; exact tested quantization repository https://huggingface.co/unsloth/gemma-4-E4B-it-GGUF ; tested revision unknown. Log reports 4.62 GiB model size and 7.52B counted parameters; size is not resident memory. Context depth, KV quantization, measured power, driver version and explicit speculation/concurrency declarations unreported. Raw source: https://blog.majid.info/llama-bench/ai-x1-pro.txt .

generation:

14.8 tokens/s

prompt processing (prefill):

212.1 tokens/s

quant:

Q4_K_XL (GGUF)

User benchmarks Qwen3.5-35B-A3B at 14.85 t/s generation and 212.05 t/s prompt processing on an AMD Radeon 890M (gfx1150) iGPU. Setup is llama.cpp (build a0ed91a44) with a Q4_K_XL GGUF, 99 layers offloaded, comparing Vulkan and ROCm backends with flash attention on and off. ROCm reaches 227.54 t/s prefill and 12.68 t/s decode without flash attention, and 229.81 t/s prefill with 13.18 t/s decode with it. A second build with unified memory and ROCm flash attention gave similar results. The same machine also ran llama-2-7b Q4_0 at 17.14 t/s decode on Vulkan and 15.52 t/s on ROCm.