llamaperf

GLM-5.3 Flash / RoCE v2 / TP2, 128 GB per node / Q4_K

on 2× AMD Strix Halo 128GB · DS4 ds4-bench-tp / ROCm Core SDK 10.0.0 / gfx1151 / 074a7ba · 4,608 ctx

AMD hardware
Oct 13, 2026
Throughput
10.1 t/s gen · 103.4 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
Q4_K (GGUF)
MTP (Multi-Token Prediction)
off
System RAM
256 GB

Summary

User reports GLM-5.3 Flash Q4_K at 10.1 t/s generation and 103.4 t/s prompt processing on two AMD Strix Halo nodes, each with 128 GB installed memory. Setup is a custom DS4 fork's staged GLM TP=2 ROCm path with a 4,608-token context allocation, 4,096-token prompt and 300-token decode at prefill batch 256, no speculative draft or MTP. The 4K point uses a cross-disciplinary prompt and is one observation; the source waived repeated timing, so no spread or confidence interval is established.