llamaperf

GLM-5.2 744B (40B active)

on AMD RX 9070 XT 16GB · colibri

Tone: mixedAMD hardware
Oct 8, 2026
Throughput
0.2 t/s gen
Quant
fmt=4 grouped int4, gs=64
System RAM
64 GB
VRAM reported
16 GB

Summary

User reports GLM-5.2 744B at 0.2414 tok/s (10-run median) on an RX 9070 XT 16GB with colibri's Vulkan backend at its default 320 resident experts. Setup is colibri's Vulkan backend with COLI_VK_EXPERTS=320, MTP drafting at 2.00 tokens per forward, fmt=4 grouped int4 weights with group size 64, experts streamed from a Gen3 x4 NVMe and 64GB DDR4 system RAM, Resizable BAR on, 40 generated tokens. The box is storage-bound. The HIP arms (0.3189 tok/s at 6 GB, 0.3331 at 12 GB) ran with MTP disabled by a guard bug, and the author has withdrawn that Vulkan-vs-HIP comparison. 640 resident experts gave 0.2475 tok/s, and with Resizable BAR off the Vulkan run fell to 0.11 tok/s.