AMD Strix Halo 128GB reports

generation:

25.0 tokens/s

prompt processing (prefill):

696.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Ministral-3-14B at 25 t/s generation and 696 t/s prompt processing on an AMD Strix Halo 128GB system with a single Radeon 8060S iGPU. Setup is llama.cpp with UD-Q4_K_XL, full offload, batch 2048 and flash attention, measured via llama-bench pp512 and tg128. The 696 t/s is native prompt processing, not TTFT-derived, and the 25 t/s is native generation throughput; the two are separate microbenchmarks, not one end-to-end request.

generation:

14.0 tokens/s

prompt processing (prefill):

716.0 tokens/s

quant:

UD-Q8_K_XL (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma-4-12B at 14 t/s generation and 716 t/s prompt processing on an AMD Strix Halo 128GB machine with a single Radeon 8060S iGPU. Setup is llama.cpp with the Unsloth UD-Q8_K_XL quant, full offload, batch 2048 and flash attention, using the RADV Vulkan backend. The 716 t/s is native PP512 prompt processing and the 14 t/s is native TG128 generation, run as separate microbenchmarks rather than one end-to-end request; the ROCm arm reports prompt processing only.

generation:

52.0 tokens/s

prompt processing (prefill):

746.0 tokens/s

quant:

Q8_0 (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Kimi-Linear-48B-A3B at 52 t/s generation and 746 t/s prompt processing on an AMD Strix Halo 128GB system with a single Radeon 8060S iGPU. Setup is llama-bench with Q8_0, full offload, batch 2048 and flash attention, using the RADV backend. The figures are separate 512-token prompt and 128-token generation microbenchmarks, not one end-to-end request; the AMDVLK backend gave 570 t/s prompt and 53 t/s generation.

generation:

38.6 tokens/s

prompt processing (prefill):

936.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Granite-4.1-8B at 38.6 t/s generation and 936 t/s prompt processing on an AMD Strix Halo 128GB system with a single Radeon 8060S iGPU. Setup is llama.cpp with UD-Q4_K_XL, full offload, batch 2048 and flash attention, using the RADV Vulkan backend. The figures are separate 512-token prompt and 128-token generation microbenchmarks, not one end-to-end request; the same GPU reached 38.7 t/s under AMDVLK.

generation:

54.0 tokens/s

prompt processing (prefill):

968.0 tokens/s

quant:

Q8_0 (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Nemotron-Cascade-2-30B-A3B at 54 t/s generation and 968 t/s prompt processing on an AMD Strix Halo 128GB system with a single Radeon 8060S iGPU. Setup is llama.cpp with Q8_0, full offload, batch 2048 and flash attention, using the RADV backend. The figures are separate llama-bench pp512 and tg128 microbenchmarks, not one end-to-end request; repetition count, context allocation and cache type are not published.

generation:

23.8 tokens/s

prompt processing (prefill):

830.8 tokens/s

quant:

Q8_0 (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Ornith-1.5-9B at 23.8 t/s generation and 830.8 t/s prompt processing on an AMD Strix Halo 128GB system with a single Radeon 8060S iGPU. Setup is llama.cpp via RADV with official Q8_0, full offload, batch 2048 and flash attention, using llama-bench pp512 and tg128. The 23.75 t/s and 830.81 t/s figures are separate 512-token prompt and 128-token generation microbenchmarks, not one end-to-end request; repetition count, context allocation and cache type are unpublished.

generation:

35.7 tokens/s

prompt processing (prefill):

972.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.5-9B at 35.7 t/s generation and 972 t/s prompt processing on a single Radeon 8060S iGPU in an AMD Strix Halo system with 128 GB unified memory. Setup is llama.cpp with UD-Q4_K_XL, full offload (ngl 999), batch 2048, 32 threads and flash attention, using the RADV Vulkan backend. The figures are separate llama-bench pp512 and tg128 microbenchmarks, not one end-to-end request; the AMDVLK backend gave 289.0 t/s prompt and 35.2 t/s generation.

generation:

73.0 tokens/s

prompt processing (prefill):

990.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports GLM-4.7-Flash at 73 t/s generation and 990 t/s prompt processing on an AMD Strix Halo 128GB system with a single Radeon 8060S iGPU. Setup is llama.cpp with UD-Q4_K_XL, full offload, batch 2048 and flash attention, measured via llama-bench pp512 and tg128. The 990 t/s and 73 t/s are separate 512-token prompt and 128-token generation microbenchmarks, not one end-to-end request; repetition count, context allocation and cache type are not published.

generation:

59.8 tokens/s

prompt processing (prefill):

1017.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.5-35B-A3B at 59.8 t/s generation and 1017 t/s prompt processing on a single Radeon 8060S iGPU in an AMD Strix Halo system with 128 GB unified memory. Setup is llama.cpp via RADV with the Unsloth UD-Q4_K_XL quant, full offload, batch 2048, flash attention and 32 threads. The figures are separate llama-bench pp512 and tg128 microbenchmarks, not one end-to-end request; the same campaign lists AMDVLK at 686.0 t/s prompt and 60.4 t/s generation.

generation:

53.8 tokens/s

prompt processing (prefill):

1010.5 tokens/s

quant:

Q8_0 (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Agents-A1 at 53.8 t/s generation and 1010.5 t/s prompt processing on a single Radeon 8060S iGPU in an AMD Strix Halo system with 128 GB unified memory. Setup is llama-bench with the official Q8_0 quant, full offload, batch 2048 and flash attention, using the RADV backend. The figures are separate 512-token prompt and 128-token generation microbenchmarks, not one end-to-end request; repetition count, context allocation and cache type are not published.

generation:

53.2 tokens/s

prompt processing (prefill):

1027.0 tokens/s

quant:

Q8_0 (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Ornith-1.0-35B at 53.2 t/s generation and 1027 t/s prompt processing on an AMD Strix Halo 128GB system with a single Radeon 8060S iGPU. Setup is llama.cpp via RADV with Q8_0, full offload, batch 2048 and flash attention. The figures are separate 512-token prompt and 128-token generation microbenchmarks, not one end-to-end request; the AMDVLK arm reached 725.0 t/s prompt and 52.5 t/s generation.

generation:

49.9 tokens/s

prompt processing (prefill):

804.2 tokens/s

quant:

Q8_0 (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Ornith-1.5-35B-A3B at 49.9 t/s generation and 804.2 t/s prompt processing on an AMD Strix Halo 128GB system with a single Radeon 8060S iGPU. Setup is llama-bench with the official Q8_0 quant, full offload, batch 2048 and flash attention, run on RADV. The figures are separate 512-token prompt and 128-token generation microbenchmarks, not one end-to-end request; repetition count, context allocation and cache type are unpublished.

generation:

59.6 tokens/s

prompt processing (prefill):

1029.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B at 59.6 t/s generation and 1029 t/s prompt processing on a single Radeon 8060S iGPU in an AMD Strix Halo system with 128 GB unified memory. Setup is llama-bench with the Unsloth UD-Q4_K_XL quant, full offload, batch 2048 and flash attention; the 20.8 GiB figure is artifact size, not measured resident memory. The rates are separate 512-token prompt and 128-token generation microbenchmarks, not one end-to-end request, and no TTFT is reported.

generation:

54.5 tokens/s

prompt processing (prefill):

1083.0 tokens/s

quant:

Q8_0 (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Nex-N2-mini at 54.5 t/s generation and 1083 t/s prompt processing on an AMD Strix Halo 128GB system with a single Radeon 8060S iGPU. Setup is llama.cpp via RADV with Q8_0, full offload, batch 2048 and flash attention. The figures are separate 512-token prompt and 128-token generation microbenchmarks, not one end-to-end request; repetition count, spread and context allocation are not published.

generation:

61.1 tokens/s

prompt processing (prefill):

1097.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Nemotron-3-Nano-Omni-30B-A3B-Reasoning at 61.1 t/s generation and 1097 t/s prompt processing on an AMD Strix Halo 128GB system with a single Radeon 8060S iGPU. Setup is llama.cpp with UD-Q4_K_XL, full offload, batch 2048 and flash attention, run natively on RADV. The figures are separate 512-token prompt and 128-token generation microbenchmarks, not one end-to-end request; the same campaign reports 58.7 t/s on AMDVLK and 56.8 t/s on ROCm.

generation:

68.0 tokens/s

prompt processing (prefill):

1106.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Nemotron-3-Nano-30B-A3B at 68 t/s generation and 1106 t/s prompt processing on an AMD Strix Halo 128GB system with a single Radeon 8060S iGPU. Setup is llama.cpp via RADV with UD-Q4_K_XL, full offload, batch 2048 and flash attention, using llama-bench pp512 and tg128. These are separate 512-token prompt and 128-token generation microbenchmarks, not one end-to-end request; the AMDVLK arm reached 776.0 and 65.0 t/s.

generation:

92.0 tokens/s

prompt processing (prefill):

1143.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3-30B-Instruct-2507 at 92 t/s generation and 1143 t/s prompt processing on an AMD Strix Halo 128GB system with a single Radeon 8060S iGPU. Setup is llama.cpp via RADV with UD-Q4_K_XL, full offload (ngl 999), batch 2048 and flash attention. The figures are separate llama-bench pp512 and tg128 microbenchmarks, not one end-to-end request; the AMDVLK run measured 936.0 t/s prompt and 93.0 t/s generation.

generation:

47.7 tokens/s

prompt processing (prefill):

1303.0 tokens/s

quant:

Q8_0 (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma-4-26B-A4B at 47.7 t/s generation and 1303 t/s prompt processing on an AMD Strix Halo 128GB system with a single Radeon 8060S iGPU. Setup is llama.cpp via RADV with Q8_0, full offload (ngl 999), batch 2048 and flash attention, using the 120 GB soft GPU-memory cap. The figures are separate llama-bench pp512 and tg128 microbenchmarks, not one end-to-end request; repetition count, spread and context allocation are not published.

generation:

37.9 tokens/s

prompt processing (prefill):

1375.0 tokens/s

quant:

Q8_0 (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.5-4B at 37.9 t/s generation and 1375 t/s prompt processing on a single Radeon 8060S iGPU in an AMD Strix Halo system with 128 GB unified memory. Setup is llama.cpp with Q8_0 and flash attention, full offload, batch 2048 and 32 threads; the source labels the run RADV Vulkan. These are separate 512-token prompt and 128-token generation microbenchmarks, not one end-to-end request. The same campaign reports AMDVLK at 510.0 t/s prompt and 40.0 t/s generation.

generation:

77.0 tokens/s

prompt processing (prefill):

1405.0 tokens/s

quant:

MXFP4 (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports GPT-OSS-20B-Derestricted at 77 t/s generation and 1405 t/s prompt processing on an AMD Strix Halo 128GB system with a single Radeon 8060S iGPU. Setup is llama.cpp with MXFP4 and full offload, batch 2048 and flash attention, using the RADV Vulkan backend. The figures are separate llama-bench pp512 and tg128 microbenchmarks, not one end-to-end request; repetition count, context allocation and cache type are not published.