llamaperf
Sep 26, 2026
Throughput
108.9 t/s gen
Quant
Q4_K (GGUF)
KV cache
FP16

Summary

User reports Ornith-1.5-35B-A3B decoding at 108.9 tok/s on an AMD MI50 32GB, 1.9x the llama.cpp baseline of 57.2 tok/s on the same GGUF and GPU. Setup is a custom HIP engine with Q4_K weights and FP16 KV cache, short context of 273-529 tokens, model load 27.6 s, sclk 1725 MHz under load. The engine reproduces llama.cpp's 8-token greedy continuation 8/8 teacher-forced and free-running; the MTP layer is not loaded.