AMD Strix Halo 128GB reports

generation:

12.1 tokens/s

prompt processing (prefill):

378.2 tokens/s

quant:

UD-Q4_K_XL (GGUF)

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B at 12.1 t/s generation and 378.2 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is llama-bench on the ROCm backend with the UD-Q4_K_XL GGUF, no speculative decoding, no chat template and no client, at depth 0. The run is a depth sweep: generation falls from 12.06 t/s at depth 0 to 9.05 t/s at depth 100000, and prompt processing from 378.24 t/s to 121.85 t/s. Each PP and TG entry is a separate phase, not one request.

generation:

48.6 tokens/s

prompt processing (prefill):

732.7 tokens/s

quant:

MXFP4 (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports gpt-oss 120B MXFP4 at 48.6 t/s generation and 732.7 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is llama.cpp build 6451 (360d6533) with the Vulkan backend, -ngl 999, flash attention enabled and -mmp 0, on a single Radeon 8060S iGPU with 128 GB unified memory. The stored figures are the FA1 comparison run's TG128 and PP512 phases, separate synthetic benchmarks rather than one request; the same run's FA0 comparison gave 49.65 t/s TG128 and 641.87 t/s PP512.

generation:

70.3 tokens/s

prompt processing (prefill):

1828.8 tokens/s

quant:

MXFP4 (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports gpt-oss 20B at 70.3 t/s generation and 1828.8 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is llama.cpp build b6451 with the Vulkan backend, MXFP4 GGUF, flash attention enabled and -ngl 999 on a single Radeon 8060S iGPU. The figures are separate synthetic PP512 and TG128 phases, not one 512-in/128-out request; a prompt sweep from PP512 to PP32768 fell from 1777.47 to 591.93 t/s, and PP65536 and PP131072 were interrupted with no completed rows.

generation:

68.4 tokens/s

prompt processing (prefill):

3095.4 tokens/s

quant:

Q6dense mixed Q6_K / UD-Q4_K_XL / Q8_0 (GGUF)

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B-Q6dense at 68.4 t/s generation and 3095.4 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is Gufo with Q6dense mixed Q6_K / UD-Q4_K_XL / Q8_0 weights, F16 K/V for prefill and an 8-bit copy for decode, single-user capacity 133760. The headline run is AR at nominal depth 4096 with 4081 cached prefix tokens and 2053 new prefill tokens, output 128; the user notes one warmed sample per point with no error bars.

generation:

67.0 tokens/s

prompt processing (prefill):

889.1 tokens/s

quant:

UD-Q4_K_XL (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports GLM-4.7-Flash at 67 t/s generation and 889.1 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is llama.cpp build 9747 with Vulkan RADV, UD-Q4_K_XL weights and f16/f16 KV cache at depth 1024, one Radeon 8060S GPU. The figures are separate synthetic PP512 and TG128 tests, not one request; at depth 16384 generation falls to 40.422869 t/s and at depth 32768 to 28.432538 t/s.

generation:

52.6 tokens/s

prompt processing (prefill):

769.8 tokens/s

quant:

UD-Q4_K_XL (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports GLM-4.7-Flash at 52.6 t/s generation and 769.8 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is llama.cpp build 1 (a66d50588b105ef81fe6c2debd19440f5a8ab97a) on ROCm with UD-Q4_K_XL and f16/f16 KV cache at depth 1024, one Radeon 8060S GPU. The headline figures are separate synthetic tests, PP512 and TG128, not one request; at depth 16384 generation falls to 35.854443 t/s and at depth 32768 to 26.583621 t/s.

generation:

66.5 tokens/s

prompt processing (prefill):

838.1 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports GLM-4.7-Flash at 66.5 t/s generation and 838.1 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is llama.cpp build 9747 with Vulkan RADV, Q4_K_M weights and f16/f16 KV cache at depth 1024, one Radeon 8060S GPU. The figures are separate synthetic tests, not one request: PP512 and TG128, each with a single repetition, so no run-to-run variance is available.

generation:

52.2 tokens/s

prompt processing (prefill):

750.2 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports GLM-4.7-Flash at 52.2 t/s generation and 750.2 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is llama.cpp build a66d50588b105ef81fe6c2debd19440f5a8ab97a on ROCm with Q4_K_M and f16/f16 KV cache at depth 1024, one Radeon 8060S GPU. The headline figures are separate synthetic tests, PP512 and TG128, not one request; at depth 32768 generation falls to 26.482385 t/s.

generation:

73.4 tokens/s

prompt processing (prefill):

1094.5 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B at 73.4 t/s generation and 1094.5 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is llama.cpp build 9747 with Vulkan RADV, Q4_K_M quantization and f16/f16 KV cache at depth 1024, one Radeon 8060S GPU. The headline figures are separate synthetic PP512 and TG128 tests, not one request; generation falls to 58.824494 t/s at depth 32768 with the same KV type.

generation:

60.0 tokens/s

prompt processing (prefill):

960.0 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B Q4_K_M at 60 t/s generation and 960 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is llama.cpp build a66d50588b105ef81fe6c2debd19440f5a8ab97a on ROCm with f16/f16 KV cache at depth 1024, one Radeon 8060S GPU, 8 threads, batch 1024 and microbatch 512. The headline figures are separate synthetic tests, PP512 and TG128, not one request; the exact values are 959.991546 t/s and 60.021451 t/s, each from a single repetition.

generation:

60.0 tokens/s

prompt processing (prefill):

1085.7 tokens/s

quant:

UD-Q4_K_XL (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen-AgentWorld-35B-A3B UD-Q4_K_XL at 60 t/s generation and 1085.7 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is llama.cpp build 9747 with Vulkan RADV, f16/f16 KV cache at depth 1024, one Radeon 8060S GPU. The headline figures are separate synthetic tests, not one request: PP512 1085.717646 t/s and TG128 60.011904 t/s, each with a single repetition.

generation:

51.2 tokens/s

prompt processing (prefill):

974.3 tokens/s

quant:

UD-Q4_K_XL (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen-AgentWorld-35B-A3B UD-Q4_K_XL at 51.2 t/s generation and 974.3 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is llama.cpp build 1 (a66d50588b105ef81fe6c2debd19440f5a8ab97a) on ROCm with f16/f16 KV cache at depth 1024, one Radeon 8060S GPU. The headline figures are separate synthetic tests, PP512 and TG128, not one request; no TTFT was measured, and each cell has a single repetition.

generation:

29.1 tokens/s

prompt processing (prefill):

201.4 tokens/s

quant:

IQ4_XS (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports MiniMax-M2.7 at 29.1 t/s generation and 201.4 t/s prompt processing on a GMKtec EVO-X2 with Ryzen AI Max+ 395 and 128 GB unified memory. Setup is llama-bench build b10605 on Vulkan with IQ4_XS, run on the integrated Radeon 8060S; the model fits wholly GPU-side. The figures are separate synthetic phases, a 512-token prompt evaluation and 128 single-token generation evaluations, not a combined request; five runs per condition were reduced to a middle-three median.

generation:

21.8 tokens/s

prompt processing (prefill):

159.4 tokens/s

quant:

Q2_K (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3-235B-A22B at 21.8 t/s generation and 159.4 t/s prompt processing on a GMKtec EVO-X2 with Ryzen AI Max+ 395 and 128 GB unified memory. Setup is llama-bench build b10605 on Vulkan with Q2_K, a 79.81 GB file, run entirely GPU-side on the integrated Radeon 8060S. The figures are separate synthetic phases, PP 512 and TG 128, each a median of five runs with the highest and lowest dropped; no TTFT was measured.

generation:

25.6 tokens/s

prompt processing (prefill):

276.9 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports GLM-4.5-Air at 25.6 t/s generation and 276.9 t/s prompt processing on a GMKtec EVO-X2 with Ryzen AI Max+ 395, integrated Radeon 8060S and 128 GB unified memory. Setup is llama-bench build b10605 on Vulkan with Q4_K_M, models fitting wholly GPU-side on the single iGPU. The figures are separate synthetic phases, PP 512 and TG 128, not a combined request; five runs per condition with the highest and lowest dropped and the middle-three median reported.

generation:

5.3 tokens/s

prompt processing (prefill):

102.0 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports llama3.3:70b at 5.3 t/s generation and 102 t/s prompt processing on a GMKtec EVO-X2 with Ryzen AI Max+ 395, integrated Radeon 8060S and 128 GB unified memory. Setup is llama-bench build b10605 with Q4_K_M over Vulkan, run on the single integrated GPU. The figures are separate synthetic phases: a 512-token prompt evaluation at 102.0 t/s and a 128 single-token generation evaluation at 5.3 t/s, not a combined request or 128 sampled output tokens. Five runs per condition were reported with the middle-three median, and no TTFT was measured.

generation:

69.6 tokens/s

prompt processing (prefill):

921.4 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Nemotron-3-Nano-30B at 69.6 t/s generation and 921.4 t/s prompt processing on a GMKtec EVO-X2 with Ryzen AI Max+ 395, Radeon 8060S iGPU and 128 GB unified memory. Setup is llama-bench build b10605 on Vulkan with Q4_K_M, run as separate 512-token prompt and 128-token generation phases. The figures are the median of five runs per condition with highest and lowest dropped; the model fits wholly GPU-side, and batch, thread and KV cache settings are not reported.

generation:

75.6 tokens/s

prompt processing (prefill):

1073.9 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Ornith-1.0-35B at 75.6 t/s generation and 1073.9 t/s prompt processing on a GMKtec EVO-X2 with Ryzen AI Max+ 395 and 128 GB unified memory. Setup is llama-bench build b10605 on Vulkan with Q4_K_M, run as separate 512-token prompt and 128-token generation phases, not a combined request. The source reports five runs per condition with the highest and lowest dropped and the middle-three median; batch, threads, ngl, flash attention and K/V types are not reported.

generation:

92.4 tokens/s

prompt processing (prefill):

1211.9 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3-Coder-30B-A3B at 92.4 t/s generation and 1211.9 t/s prompt processing on a GMKtec EVO-X2 with Ryzen AI Max+ 395, Radeon 8060S iGPU and 128 GB unified memory. Setup is llama-bench build b10605 on Vulkan with Q4_K_M, models fitting wholly GPU-side. The figures are separate synthetic phases, PP 512 and TG 128, not a combined request; five runs per condition with the middle-three median reported, and no TTFT measured.

generation:

24.5 tokens/s

prompt processing (prefill):

728.5 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3 14B at 24.5 t/s generation and 728.5 t/s prompt processing on a GMKtec EVO-X2 with Ryzen AI Max+ 395, integrated Radeon 8060S and 128 GB unified memory. Setup is llama-bench build b10605 with Q4_K_M on Vulkan, run on the single integrated GPU. The figures are separate synthetic phases: 512-token prompt evaluation and 128 single-token generation evaluations, not a combined request. Five runs per condition were reported, with the middle-three median after dropping the highest and lowest.