Qwen2.5 7B

on AMD Ryzen AI Max+ 395 / Radeon 8060S (Framework Desktop; 64 GB unified) · llama.cpp

Oct 13, 2026

Throughput

47.7 t/s gen · 1512.4 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.

Quant

Q4_K_M (GGUF)

KV cache

F16

Flash Attention

MTP (Multi-Token Prediction)

off

System RAM

64 GB

Summary

User reports Qwen2.5-7B-Instruct Q4_K_M at 47.7 t/s generation and 1512.4 t/s prompt processing on a Framework Desktop with an AMD Ryzen AI Max+ 395 and Radeon 8060S. Setup is llama.cpp build baf6360be with f16 K/V cache, flash attention, n_batch and n_ubatch of 512, and one Vulkan device. The figures are separate depth-0 arms: PP512 at 1512.379830 t/s and TG32 at 47.677208 t/s, not one 512-input/32-output run. At depth 32768 the same arm drops to 662.571662 t/s PP512 and 33.575848 t/s TG32.