llama.cpp on the AMD Strix Halo 128GB

On the AMD Strix Halo 128GB, llama.cpp runs Qwen3.8 125B · 6B active at 4-bit at a median 23.0 tokens per second for generation, from 3 plain runs (128K to 160K context, 1 not stated).

Everything reported on the AMD Strix Halo 128GB llama.cpp on other cards What fits on the AMD Strix Halo 128GB

Speed by model on the AMD Strix Halo 128GB

Plain runs only: one device, one request, no speculative decoding, the whole model in memory. Each row is one model size at one bit class, with the other engines' plain runs of the same model on this card beside it. Context length and build still differ between runs, and each side shows its own.

Model	llama.cpp	Other engines
Qwen3.8 125B · 6B active 4-bit	23.0 t/smedian of 3, 128K to 160K context, 1 not stated	Gufo 26.0 t/smedian of 3, 132K to 134K context, 1 not stated
Qwen3 30B · 3B active 4-bit	72.3 t/smedian of 2, 48K context, 1 not stated	no other engine reported
Qwen3.6 35B · 3B active 4-bit	51.2 t/smedian of 2, 30K context, 1 not stated	no other engine reported
Qwen3.8 27B 4-bit	14.0 t/smedian of 2, 32K context, 1 not stated	no other engine reported
gpt-oss 117B · 5.1B active 4-bit	54.1 t/sone report, context not stated	no other engine reported
LFM2.5 2.6B 4-bit	113.0 t/sone report, 125K context	no other engine reported
Mimo 2.6 309B · 15B active 2-bit	26.1 t/sone report, 32K context	no other engine reported
Ornith 1.0 35B · 3B active 4-bit	74.1 t/sone report, context not stated	no other engine reported
Ornith1.5 35B · 3B active 4-bit	72.6 t/sone report, context not stated	no other engine reported
Qwen3-Coder 30B · 3B active 4-bit	70.3 t/sone report, 8K context	no other engine reported
Qwen3-Coder-Next 80B · 3B active 6-bit	36.8 t/sone report, 256K context	no other engine reported
Qwen3.5 122B · 10B active 6-bit	19.2 t/sone report, context not stated	no other engine reported
Qwen3.5 35B · 3B active 4-bit	45.0 t/sone report, 128K context	no other engine reported
Qwen3.5 35B · 3B active 8-bit	49.7 t/sone report, context not stated	no other engine reported
Qwen3.6 35B · 3B active 6-bit	50.0 t/sone report, context not stated	no other engine reported
Qwen3.6 35B · 3B active 8-bit	24.1 t/sone report, 512K context, 1 not stated	Gufo 50.8 t/sone report, context not stated
Qwen3.8 27B 8-bit	7.2 t/sone report, context not stated	no other engine reported

91 llama.cpp reports on the AMD Strix Halo 128GB

Newest first. A run on several cards, with speculative decoding, with part of the model in system RAM or with batched requests says so, since each describes a different setup.

gpt-oss · MXFP451.1 t/s
Llama 3.2 · Q6_K_L69.0 t/s
Qwen3-Coder · Q4_K_M69.6 t/s
Qwen3-Coder · UD-Q4_K_XL65.9 t/s
Qwen3-Coder · UD-Q4_K_XL59.8 t/s
Qwen3-Coder · UD-Q4_K_XL71.3 t/s
Qwen3-Coder · UD-Q4_K_XL73.0 t/s
Qwen3.8 · Q4_K_XL12.2 t/s
DeepSeek V4 Flash · IQ3_XXS18.6 t/s
Muse · Q4_K_M13.2 t/s
Model not identified · Q4_K_M80.4 t/s
Nemotron 3.5 · Q4_066.8 t/s
Qwen3.6 · Q8_054.2 t/s
Qwen3.8 · Q8_07.8 t/s
Llama 3.2 · Q4_K_M85.1 t/s
gpt-oss · MXFP473.0 t/s
Qwen3 · Q4_K_M73.7 t/s
Qwen3.6 · Q4_K_M12.1 t/s
Qwen3.6 · BF1621.5 t/s
Qwen3.6 · UD-Q4_K_M60.4 t/s
Qwen3.6 · UD-Q4_K_XLPP 148.8 t/s
Qwen3.6 · UD-Q4_K_XLPP 32.1 t/s
Qwen3.6 · Q4_K_M64.0 t/s
Gemma 3 · Q4_013.0 t/s
Qwen3.8 · UD-Q2_K_XL24.9 t/s
Qwen3.8 · UD-Q2_K_XL24.3 t/s
Qwen3.8 · UD-Q4_K_XL20.9 t/s
Qwen3.8 · UD-Q4_K_XL21.2 t/s
Qwen3.8 · UD-Q4_K_XL21.0 t/s
Ornith1.5 35B · 3B active · Q6_K50.1 t/s

Filter these reports on the homepage

Frequently asked

How fast is llama.cpp on the AMD Strix Halo 128GB?

On the AMD Strix Halo 128GB, llama.cpp runs Qwen3.8 125B · 6B active at 4-bit at a median 23.0 tokens per second for generation, from 3 plain runs (128K to 160K context, 1 not stated).

Is llama.cpp faster than other engines on the AMD Strix Halo 128GB?

Same card, same model size and bit class, plain runs only: Qwen3.8 125B · 6B active at 4-bit gives llama.cpp 23.0 t/s (3 runs) and Gufo 26.0 t/s (3). Context length and engine build differ between the runs, so treat it as a lead, not a benchmark.