llamaperf

Inkling-2 Mini

on AMD Strix Halo 128GB · Gufo

Tone: positiveAMD hardware
Nov 21, 2026
Throughput
29-30 t/s gen · 650.0 tokens/s prompt processing (prefill)Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.
Quant
UD-Q4_K_M (GGUF)
System RAM
128 GB

Use cases

visiontool-uselong-context

Summary

User reports Inkling-2-Mini decoding at ~29-30 tok/s on Strix Halo with a from-scratch HIP engine written as a Gufo extension, against ~25 tok/s for llama.cpp on the same machine. Setup is Gufo with UD-Q4_K_M GGUF weights in the Mainline_Support layout, single machine on gfx1151; prefill is ~650 tok/s and decode falls to 26.5 tok/s at 20k depth. Continuous batching gives 56 tok/s aggregate across 4 concurrent requests versus 30 single. DSpark speculative decoding is lossless but slower at 17.6 tok/s with 10-30% acceptance. A 31k-token needle test took 71 s versus 255 s for llama.cpp. The author notes the hyperloom implementation is about 50% faster, placing this between llama.cpp and loom.

Compare this run

Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.

Models most run on the AMD Strix Halo 128GB

Bars are medians on one scale, 0 to 77.0 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.

Everything reported on the AMD Strix Halo 128GB →Every Inkling-2 report →Estimate Inkling-2 on this card →