AMD Strix Halo 128GB reports

generation:

92.4 tokens/s

prompt processing (prefill):

1211.9 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3-Coder-30B-A3B at 92.4 t/s generation and 1211.9 t/s prompt processing on a GMKtec EVO-X2 with Ryzen AI Max+ 395, Radeon 8060S iGPU and 128 GB unified memory. Setup is llama-bench build b10605 on Vulkan with Q4_K_M, models fitting wholly GPU-side. The figures are separate synthetic phases, PP 512 and TG 128, not a combined request; five runs per condition with the middle-three median reported, and no TTFT measured.

generation:

24.5 tokens/s

prompt processing (prefill):

728.5 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3 14B at 24.5 t/s generation and 728.5 t/s prompt processing on a GMKtec EVO-X2 with Ryzen AI Max+ 395, integrated Radeon 8060S and 128 GB unified memory. Setup is llama-bench build b10605 with Q4_K_M on Vulkan, run on the single integrated GPU. The figures are separate synthetic phases: 512-token prompt evaluation and 128 single-token generation evaluations, not a combined request. Five runs per condition were reported, with the middle-three median after dropping the highest and lowest.

generation:

39.1 tokens/s

prompt processing (prefill):

1073.8 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Ornith-1.0-9B at 39.1 t/s generation and 1073.8 t/s prompt processing on a GMKtec EVO-X2 with Ryzen AI Max+ 395 and 128 GB unified memory. Setup is llama-bench build b10605 on Vulkan with Q4_K_M, run on the integrated Radeon 8060S iGPU. The figures are separate synthetic phases: a 512-token prompt evaluation and a 128 single-token generation evaluation, not a combined request. The source reports five runs per condition, drops the highest and lowest, and gives the middle-three median.

generation:

41.0 tokens/s

prompt processing (prefill):

1191.0 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3 8B at 41 t/s generation and 1191 t/s prompt processing on a GMKtec EVO-X2 with Ryzen AI Max+ 395, Radeon 8060S iGPU and 128 GB unified memory. Setup is llama-bench build b10605 with Q4_K_M and Vulkan, run as separate 512-token prompt and 128-token generation phases rather than one request. The source reports five runs per condition, dropping the highest and lowest for a middle-three median, and says the model fits wholly GPU-side. Batch, thread, context and KV cache settings are not reported.

generation:

69.5 tokens/s

prompt processing (prefill):

1392.6 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Nemotron-3-Nano-4B at 69.5 t/s generation and 1392.6 t/s prompt processing on a GMKtec EVO-X2 with Ryzen AI Max+ 395, Radeon 8060S iGPU and 128 GB unified memory. Setup is llama-bench build b10605 with Q4_K_M over Vulkan, run as separate 512-token prompt and 128-token generation phases rather than one request. The source reports five runs per condition, dropping the highest and lowest for a middle-three median, and says the model fits wholly GPU-side; batch, thread and context settings are not reported.

generation:

79.7 tokens/s

prompt processing (prefill):

2368.4 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports phi4-mini Q4_K_M at 79.7 t/s generation and 2368.4 t/s prompt processing on a GMKtec EVO-X2 with Ryzen AI Max+ 395, Radeon 8060S iGPU and 128 GB unified memory. Setup is llama-bench build b10605 on Vulkan, with the model fitting wholly GPU-side. The figures are separate synthetic phases: a 512-token prompt evaluation and a 128 single-token generation evaluation, not a combined request or 128 sampled output tokens. Five runs per condition were reported with the middle-three median, and no TTFT was measured.

generation:

158.0 tokens/s

prompt processing (prefill):

7107.3 tokens/s

quant:

Q8_0 (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports llama3.2:1b at 158 t/s generation and 7107.3 t/s prompt processing on a GMKtec EVO-X2 with Ryzen AI Max+ 395 and 128 GB unified memory. Setup is llama-bench build b10605 on the integrated Radeon 8060S via Vulkan, with Q8_0 and the model fitting wholly GPU-side. The figures are separate synthetic phases: a 512-token prompt evaluation and 128 single-token generation evaluations, not a combined request or 128 sampled output tokens. Five runs per condition were reported, with the highest and lowest dropped and the middle-three median given.

generation:

23.1 tokens/s

prompt processing (prefill):

1130.6 tokens/s

quant:

Q8_0 (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.5-9B at 23.1 t/s generation and 1130.6 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is llama.cpp's llama-bench with Q8_0 and -ngl 999, run through docker compose on ROCm 7.2.1. The two figures come from separate synthetic phases: PP 512 at 1130.63 +/- 21.38 t/s and TG 128 at 23.11 +/- 0.03 t/s, where the second phase is 128 single-token steps rather than a real 128-token answer. No TTFT was measured.

generation:

11.3 tokens/s

prompt processing (prefill):

295.2 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.5-27B at 11.3 t/s generation and 295.2 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is llama.cpp via docker compose with the Q4_K_M GGUF at -ngl 999, stock F16 K/V cache and flash attention off. The two phases are separate synthetic runs: PP 512 at 295.17 +/- 24.78 t/s and TG 128 at 11.27 +/- 0.01 t/s, where the second is 128 single-token steps rather than a 128-token answer. No TTFT was measured.

generation:

7.5 tokens/s

prompt processing (prefill):

296.9 tokens/s

quant:

Q8_0 (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.5-27B at 7.5 t/s generation and 296.9 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is llama.cpp's llama-bench with Q8_0 and F16 K/V cache, run with -ngl 999 on a Radeon 8060S. The two phases are separate: PP 512 at 296.89 +/- 24.46 t/s and TG 128 at 7.49 +/- 0.01 t/s, where the second is 128 synthetic single-token steps rather than a real 128-token answer.

generation:

44.0 tokens/s

prompt processing (prefill):

1079.9 tokens/s

quant:

Q8_0 (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.5-35B-A3B at 44 t/s generation and 1079.9 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is llama.cpp's llama-bench with the Q8_0 GGUF at -ngl 999, stock five repeats, depth 0, batch 2048 / ubatch 512, F16 K/V, flash attention off and mmap on. The two phases are separate: PP 512 at 1079.88 +/- 55.12 t/s and TG 128 at 44.04 +/- 0.12 t/s, where the second is 128 synthetic single-token steps rather than a real 128-token answer, and no TTFT was measured.

generation:

23.4 tokens/s

prompt processing (prefill):

1263.5 tokens/s

quant:

Q4 Mix (Q4_K/Q5_1/IQ4_NL/Q8_0) (GGUF)

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next-Uncensored at 23.4 t/s generation and 1263.5 t/s prompt processing on an AMD Strix Halo 128GB system. Setup is the Gufo runtime over ROCm/HIP with a Q4 Mix GGUF (Q4_K/Q5_1/IQ4_NL/Q8_0) and autoregressive decoding, MTP off, one GPU. The figures are medians of three mixed-long requests at 10,583 input and 128 output tokens each, with all prompts uncached; client-observed TTFT median was 8.436 seconds.

generation:

26.0 tokens/s

prompt processing (prefill):

1489.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next at 26 t/s generation and 1489 t/s prompt processing on an AMD Strix Halo 128GB box. Setup is Gufo 0.12.0 with ROCm/HIP, UD-Q4_K_XL GGUF, on the single Radeon iGPU; the bench path accepts only concurrency 1. The figures are separate native microbenchmarks, PP4096 at 1489 t/s and TG128 at 26 t/s, not one 4096-in/128-out request; the 32768-token prompt phase is recorded separately at 1400 t/s.

generation:

23.6 tokens/s

prompt processing (prefill):

321.8 tokens/s

quant:

UD-Q4_K_XL (GGUF)

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next at 23.6 t/s generation and 321.8 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is llama.cpp with the Unsloth UD-Q4_K_XL GGUF, offloaded fully to the iGPU, at a prepared depth of 32768 with no MTP. The 321.8/23.6 figures come from a PP512/TG32 run with Nathanw v0.7.5, compared against stock pr28243 at 215.6/20.3 tokens/s; TG32 and TG8 results are not cross-paired.

generation:

31.7 tokens/s

prompt processing (prefill):

935.0 tokens/s

quant:

IQ4_NL-PROJFIX + shared Q8_0 MTP sidecar (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next at 31.7 t/s generation and 935 t/s prompt processing on a single Radeon 8060S gfx1151 in an AMD Ryzen AI MAX+ 395 with 128 GB unified memory. Setup is llama.cpp with IQ4_NL-PROJFIX and a shared Q8_0 MTP sidecar, FP16 KV cache, one slot, batch/microbatch 16384, MTP width 3, at 131072 tokens of context. The 131072-token run used 122869 input and 512 output tokens with 131.48 s TTFT; the source also lists 196608- and 262144-token runs at 29.09 and 28.49 t/s.

generation:

72.0 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Kimi-Linear-48B-A3B at 72 t/s generation on one Radeon 8060S gfx1151 in an AMD Ryzen AI MAX+ 395 with 128 GB unified memory. Setup is native llama-bench with Q4_K_M, full GPU offload at ngl 999, batch 2048, 32 threads and flash attention 1. The 72 t/s is the tg128 phase, not simultaneous 512-in/128-out throughput; the source compares it to 53 t/s for the separate Q8_0 phase.

generation:

22.5 tokens/s

quant:

Q8_0 (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.5-9B at 22.5 t/s generation on a single Radeon 8060S gfx1151 in an AMD Ryzen AI MAX+ 395 with 128 GB unified memory. Setup is native llama-bench with Q8_0 weights, full GPU offload, batch 2048, 32 threads and flash attention 1, measured on the RADV backend. The figure is the tg128 phase, 128 generated tokens, not simultaneous prompt and generation throughput; the same model reached 23.3 t/s on AMDVLK and 22.3 t/s on ROCm.

generation:

52.9 tokens/s

prompt processing (prefill):

1196.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma-4-26B-A4B at 52.9 t/s generation and 1196 t/s prompt processing on a single Radeon 8060S gfx1151 in an AMD Ryzen AI MAX+ 395 with 128 GB unified memory. Setup is llama-bench with UD-Q4_K_XL, full GPU offload at ngl 999, batch 2048, 32 threads and flash attention 1. The rates come from separate pp512 and tg128 phases, not simultaneous 512-in/128-out throughput; the 16 GiB artifact is distinct from the Q8_0 25 GiB table.

prompt processing (prefill):

259.0 tokens/s

quant:

AesSedai Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.5-122B-A10B at 259 t/s prompt processing on a single Radeon 8060S gfx1151 in an AMD Ryzen AI MAX+ 395 with 128 GB unified memory. Setup is the RADV backend with AesSedai Q4_K_M, full GPU offload at ngl 999, batch 2048, 32 threads and flash attention 1. The figure is a PP-only measurement over 512 prompt tokens; no TG result was supplied for this configuration.

prompt processing (prefill):

832.0 tokens/s

quant:

AesSedai Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.5-35B-A3B at 832 t/s prompt processing on a single Radeon 8060S gfx1151 in an AMD Ryzen AI MAX+ 395 with 128 GB unified memory. Setup is native llama-bench with AesSedai Q4_K_M, full GPU offload, batch 2048, 32 threads and flash attention 1; the 21 GiB artifact is disk size, not measured GPU memory. The 832 t/s is the RADV pp512 phase; the AMDVLK tg128 phase reached 53 t/s, and PP and TG are separate phases rather than simultaneous throughput.