llamaperf

Inkling-2 small

on AMD Strix Halo 128GB · libosmos

Tone: positiveAMD hardware
Nov 16, 2026
Throughput
14-24 t/s gen
Quant
UD-IQ1_S (GGUF)

Summary

User reports Inkling-2-small running on a single AMD Strix Halo at 14-24 tok/s decode with drafting and over 310 tok/s prefill. Setup is libosmos v0.8.0 with a patch pack based on the Unsloth UD-IQ1_S quant, about 156GB. The pack reports a KLD of 0.196 and top-1 agreement of 85.8%, with more optimizations promised in later versions.