llamaperf

GLM-5.3 REAP144 Coder

on AMD RX 7900 XTX · llama.cpp · 384,000 ctx

Tone: mixedAMD hardware
Oct 14, 2026
Throughput
16.0 t/s gen
Quant
IQ2_XXS (GGUF)
System RAM
32 GB
VRAM reported
24 GB

Use cases

codingagenticlong-context

Summary

User pruned GLM-5.3-Flash (321B MoE) from 288 to 144 routed experts per layer, selecting experts on agentic coding sessions, then quantized with an imatrix built on the same sessions. The IQ2_XXS build (45.3 GB) runs on an RX 7900 XTX (24 GB) plus 32 GB DDR4-2133 with an i7-7700K over PCIe 3.0, at roughly 16 tok/s generation and roughly 950 tok/s prompt processing. Setup is llama.cpp with IQ2_XXS GGUF weights and 384K context. The 2-bit version is experimental: it solved 39/48 attempts versus 48/48 for Q3_K and above, and when it fails it loops or keeps exploring until the turn limit. An anti-loop guard did not help (27/37 vs 30/37). Evaluation used 26 Python SWE tasks from ansible, openlibrary and qutebrowser, 2 attempts each, 200 turns, in a sealed clean room with no network or git history. The code-scored expert mask needed x1.21 turns versus the full model, against x2.21 for a frequency mask and x2.29 for the public multi-domain REAP 50% mask. IQ2_S wants 24 + 48 GB, Q3_K 24 + 64 GB, Q4_K_M 24 + 96 GB or 128 GB unified memory.