Gemma 4 26B (4B active)
on AMD Radeon AI PRO R9700 32GB · llama.cpp
Summary
User reports gemma-4-26B-A4B Q8_0 (26.9 GB) on a Radeon AI PRO R9700 32GB in a Minisforum DEG2 eGPU over USB4, at about 3,200 t/s prefill (pp512) and 66-71 t/s decode, versus 1,387 t/s and 46.6 t/s on the Strix Halo iGPU.
Setup is llama.cpp Vulkan/RADV with -ngl 99 -fa 1, a 256 MB BAR, the card pinned to profile_peak, and both GGML_VK_DISABLE_HOST_VISIBLE_VIDMEM=1 and GGML_VK_MMID_F16B=0 set. A 10 h soak ran 6,724/6,724 requests with decode flat from 66.2 to 63.7 tok/s between the first and last quarter.
Big model loads wedge the card at the default power level unless profile_peak is set, which idles at 137 W; a wrapper switching it around jobs passed 22/22 image jobs. A full 32 GB BAR works only with the dock powered on at boot, where 27B Q8_0 decode is 17.9 tok/s at a 1k context.