llamaperf

GLM-5.3 321B (18B active) Flash / Maya-S-v2

on AMD Strix Halo 128GB · Project Maya · 262,144 ctx

AMD hardware
Oct 11, 2026
Throughput
18.3 t/s gen · 320.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
Maya-S-v2-IQ2_XXS (GGUF)
System RAM
128 GB

Summary

User reports GLM-5.3 Flash (Maya-S-v2 IQ2_XXS) at 18.3 t/s generation and 320 t/s prompt processing on an AMD Strix Halo 128GB. Setup is Project Maya 1.0.30 on HIP with the Maya-S-v2-IQ2_XXS mixed quant, one request, 262K configured context, the whole model in unified memory; whether MTP was on is not stated. Source: https://github.com/mw00/project-maya/issues/102 (LIN-LAB-AI, 2026-10-11; third-party measurement, not our run). Headline metrics are separate built-in --bench tests: 320 tok/s prefill for 8,192 input tokens; generation 18.3 tok/s reported mean, three 256-token answers (18.3/18.4/18.3), greedy, thinking off. Short/warm decode is not generation following that 8K prefill. Also: 267 tok/s prefill for 2,048 input tokens. Prefill repetition count unspecified. Configured context 262,144, not headline prompt length. Hardware/runtime: one Radeon 8060S (gfx1151), Ryzen AI Max+ 395, 128GB unified; 96GiB BIOS carve-out. Windows reports 31.65GiB host RAM; HIP reports 107.87GB GPU memory. These are overlapping unified-memory views, not additive capacities. Native Windows 11 build 26100; HIP SDK 7.2/clang21; Maya 1.0.30 c189f7a; --backend hip --gpu 0; NVMe model storage; --prefill auto, PREFILL_SUB=1024, PREFILL_MB=6144. Separate cold technical-prose sweep asserts reused=0 and uses engine prompt_ms/decode_ms: at 8,632–9,070 input tokens median 315.1 pp / 15.90 tg; at 261,748–261,771 input median 274.5 pp / 14.20 tg. Answers usually ~65 tokens, so not comparable to 256-token headline. No downloadable per-request sweep logs linked. KV precision, Flash Attention, MTP status, clocks/power and exact device manufacturer unspecified. Verified model identity: peasantsmith/GLM-5.3-Flash-Maya-GGUF, folder Maya-S-v2-IQ2_XXS, three GLM-5.3-Flash-Maya-S-v2-IQ2_XXS-0000{1,2,3}-of-00003.gguf shards. Source reports SHA256 verification against its launcher. Tested-version manifest: https://github.com/mw00/project-maya/blob/v1.0.30/maya.py . It accepts two first-shard headers and uses revision main; exact tested Hugging Face commit is unknown. Quant is mixed: IQ2_XXS gate/up experts, IQ2_S/IQ3_XXS down, Q6_K attention, including an MTP block (presence does not establish use). Model card/file verification: https://huggingface.co/peasantsmith/GLM-5.3-Flash-Maya-GGUF/tree/main/Maya-S-v2-IQ2_XXS and https://huggingface.co/peasantsmith/GLM-5.3-Flash-Maya-GGUF . Card identifies 321B total/~18B active and 96.5GB text weights. Source 90.90GiB on-disk includes vision files; not peak RAM. Vision was untested. No quality benchmark reported.