Q4_K

on 2× AMD Strix Halo 128GB · DS4 ds4-bench-tp / ROCm 7.14 / gfx1151 / 71e6a24

Oct 13, 2026

Throughput

9.8 t/s gen · 97.9 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.

Quant

Q4_K (GGUF)

MTP (Multi-Token Prediction)

off

System RAM

256 GB

Summary

User reports GLM-5.3 at 9.8 t/s generation and 97.9 t/s prompt processing on two AMD Strix Halo nodes with 128GB each. Setup is the staged GLM path over OdinLink USB4/TB5 RDMA with Q4_K, TP=2 across two Radeon 8060S iGPUs, fixed 4,096 prompt tokens and 300 generated tokens at prefill batch 256. The source gives exact rates of 97.89 t/s prefill and 9.77 t/s decode from a single diagnostic run, with no TTFT reported and no single-node scaling claimed.