AMD Strix Halo 128GB reports

generation:

10.2 tokens/s

prompt processing (prefill):

196.0 tokens/s

quant:

Unsloth UD-Q4_K_XL (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Nemotron-3-Super-120B-A12B at 10.2 t/s generation and 196 t/s prompt processing on an AMD Strix Halo 128GB system with one Radeon 8060S gfx1151 integrated GPU. Setup is native llama-bench with Unsloth UD-Q4_K_XL, full GPU offload ngl 999, batch 2048, 32 threads and flash attention 1. The figures are separate pp512 and tg128 phases, not simultaneous 512-in/128-out throughput; the same model also ran at 9.86 t/s TG and 139 t/s PP on the AMDVLK backend.

generation:

32.0 tokens/s

prompt processing (prefill):

237.0 tokens/s

quant:

IQ3_XS (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Step3.5-Flash at 32 t/s generation and 237 t/s prompt processing on an AMD Strix Halo 128GB system with one Radeon 8060S gfx1151 integrated GPU. Setup is the RADV backend with IQ3_XS weights, full GPU offload, batch 2048, 32 threads and flash attention 1. The figures come from separate pp512 and tg128 phases, not simultaneous 512-in/128-out throughput; the IQ3_XS artifact is 76 GiB on disk.

generation:

2.9 tokens/s

prompt processing (prefill):

62.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Mistral-Medium-3.5-128B at 2.9 t/s generation and 62 t/s prompt processing on an AMD Strix Halo 128GB system with a single Radeon 8060S iGPU. Setup is llama-bench with the Unsloth UD-Q4_K_XL quant, full offload, batch 2048 and flash attention; the source lists a 120 GB soft GPU-memory cap. The 2.9 t/s is the RADV Vulkan figure; ROCm gave 2.92 t/s and AMDVLK 2.99 t/s, while the main table lists both Vulkan runs as 2.9. These are separate 512-token prompt and 128-token generation microbenchmarks, not one end-to-end request.

generation:

18.3 tokens/s

prompt processing (prefill):

147.5 tokens/s

quant:

UD-IQ3_XXS (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports DeepSeek-V4-Flash-0731 at 18.3 t/s generation and 147.5 t/s prompt processing on a single Radeon 8060S iGPU in an AMD Strix Halo system with 128 GB unified memory. Setup is llama.cpp b11093 via RADV Vulkan with the Unsloth UD-IQ3_XXS quant, full offload, batch 2048 and flash attention. The figures are separate llama-bench pp512 and tg128 microbenchmarks, not one end-to-end request; an earlier RADV run measured 114.7/12.4 t/s with missing kernels.

generation:

7.3 tokens/s

prompt processing (prefill):

234.1 tokens/s

quant:

UD-Q8_K_XL (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Muse-Glimmer-30B at 7.3 t/s generation and 234.1 t/s prompt processing on a single Radeon 8060S iGPU in an AMD Strix Halo system with 128 GB unified memory. Setup is llama.cpp with the Unsloth UD-Q8_K_XL quant, full offload, batch 2048 and flash attention, run as separate llama-bench pp512 and tg128 microbenchmarks rather than one end-to-end request. The main table figure is rounded; the source narrative gives approximately PP234 and TG7.25. A separate DFlash server test reports 13.53 t/s, which is not tg128 and is not part of this row.

generation:

7.5 tokens/s

prompt processing (prefill):

236.7 tokens/s

quant:

Q8_0 (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B at 7.5 t/s generation and 236.7 t/s prompt processing on a single Radeon 8060S iGPU in an AMD Strix Halo system with 128 GB unified memory. Setup is llama.cpp with official Q8_0, full offload (ngl 999), batch 2048, flash attention and 32 threads. The figures are separate llama-bench pp512 and tg128 microbenchmarks, not one end-to-end request; the home narrative reports 7.53 t/s for the same run.

generation:

22.0 tokens/s

prompt processing (prefill):

179.0 tokens/s

quant:

UD-Q3_K_XL (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports MiniMax-M2.5 at 22 t/s generation and 179 t/s prompt processing on an AMD Strix Halo 128GB machine with a single Radeon 8060S iGPU. Setup is llama.cpp via RADV with the Unsloth UD-Q3_K_XL quant, full offload, batch 2048 and flash attention. The rates are separate llama-bench pp512 and tg128 microbenchmarks, not one end-to-end request; the same campaign lists AMDVLK at 164.0 and 32.0 t/s.

generation:

27.0 tokens/s

prompt processing (prefill):

181.0 tokens/s

quant:

UD-IQ4_XS (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports MiniMax-M2.7 at 27 t/s generation and 181 t/s prompt processing on a single Radeon 8060S iGPU in an AMD Strix Halo 128GB system. Setup is llama.cpp with the Unsloth UD-IQ4_XS quant, full offload, batch 2048 and flash attention, using the RADV backend. The rates are separate llama-bench pp512 and tg128 microbenchmarks, not one end-to-end request; repetition count, context allocation and cache type are not published.

generation:

11.1 tokens/s

prompt processing (prefill):

261.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma-4-31B at 11.1 t/s generation and 261 t/s prompt processing on an AMD Strix Halo 128GB system with a single Radeon 8060S iGPU. Setup is llama.cpp via RADV with the Unsloth UD-Q4_K_XL quant, full offload, batch 2048 and flash attention. The 261 t/s is native PP512 and the 11.1 t/s is native TG128, separate microbenchmarks rather than one end-to-end request; the same 11.1 t/s generation is reported for AMDVLK.

generation:

21.6 tokens/s

prompt processing (prefill):

263.0 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Laguna S 2.1 at 21.6 t/s generation and 263 t/s prompt processing on a single Radeon 8060S iGPU in an AMD Strix Halo 128GB system. Setup is llama.cpp with the Poolside Q4_K_M quant, full offload, batch 2048, 32 threads and flash attention enabled. The figures are separate 512-token prompt and 128-token generation microbenchmarks, not one end-to-end request; the RADV arm ran 263.0 t/s prompt and 21.63 t/s generation.

generation:

22.7 tokens/s

prompt processing (prefill):

323.5 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Puzzle 75B-A9B at 22.7 t/s generation and 323.5 t/s prompt processing on an AMD Strix Halo 128GB system with a single Radeon 8060S iGPU. Setup is llama.cpp with the YanissAmz Q4_K_M quant, full offload, batch 2048, flash attention and 10 threads under ROCm. The figures are separate 512-token prompt and 128-token generation microbenchmarks, not one end-to-end request; a separate ROCm thread sweep reports 10 threads at 22.14 t/s.

generation:

11.8 tokens/s

prompt processing (prefill):

275.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Granite-4.1-30B at 11.8 t/s generation and 275 t/s prompt processing on an AMD Strix Halo 128GB system with a single Radeon 8060S iGPU. Setup is llama.cpp with UD-Q4_K_XL, full offload, batch 2048 and flash attention, measured via llama-bench pp512 and tg128. The 275 t/s is native prompt processing, not TTFT, and the 11.8 t/s is native generation throughput; repetition count, context allocation and cache type are not published.

generation:

22.4 tokens/s

prompt processing (prefill):

287.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.5-122B-A10B at 22.4 t/s generation and 287 t/s prompt processing on a single Radeon 8060S iGPU in an AMD Strix Halo system with 128 GB unified memory. Setup is llama.cpp with the Unsloth UD-Q4_K_XL quant, full offload, batch 2048 and flash attention, measured via llama-bench pp512 and tg128. The 287 t/s and 22.4 t/s are separate 512-token prompt and 128-token generation microbenchmarks, not one end-to-end request; the 72 GiB figure is artifact disk size, not measured resident memory.

generation:

12.1 tokens/s

prompt processing (prefill):

310.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.5-27B at 12.1 t/s generation and 310 t/s prompt processing on an AMD Strix Halo 128GB system with a single Radeon 8060S iGPU. Setup is llama.cpp with UD-Q4_K_XL, full offload, batch 2048 and flash attention, using the RADV Vulkan backend. The figures are separate 512-token prompt and 128-token generation microbenchmarks, not one end-to-end request; the AMDVLK backend gave 86.0 t/s prompt and 11.9 t/s generation.

generation:

12.0 tokens/s

prompt processing (prefill):

322.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.6-27B at 12 t/s generation and 322 t/s prompt processing on a single Radeon 8060S iGPU in an AMD Strix Halo system with 128 GB unified memory. Setup is llama-bench with the Unsloth UD-Q4_K_XL quant, full offload, batch 2048 and flash attention; the 120 GB GPU-memory cap is a software limit, not observed usage. The 12 t/s and 322 t/s are separate 128-token generation and 512-token prompt microbenchmarks, not one end-to-end request; the narrative cites 12.03 t/s while the table rounds to 12.0.

generation:

40.3 tokens/s

prompt processing (prefill):

363.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Mistral-Small-4-119B at 40.3 t/s generation and 363 t/s prompt processing on a single Radeon 8060S iGPU in an AMD Strix Halo system with 128 GB unified memory. Setup is llama.cpp Vulkan build 9518 with Unsloth UD-Q4_K_XL, full offload, batch 2048 and flash attention. The figures are separate 512-token prompt and 128-token generation microbenchmarks, not one end-to-end request; the quant is split across three files totalling 69 GiB.

generation:

15.0 tokens/s

prompt processing (prefill):

382.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Devstral-Small-2-24B at 15 t/s generation and 382 t/s prompt processing on an AMD Strix Halo 128GB system with a single Radeon 8060S iGPU. Setup is llama-bench with UD-Q4_K_XL, full offload, batch 2048 and flash attention; the 120 GB GPU-memory cap is a software limit, not observed usage. The 382 t/s is native PP512 and the 15 t/s is native TG128, separate microbenchmarks rather than one end-to-end request; repetition count, spread and context allocation are not published.

generation:

15.0 tokens/s

prompt processing (prefill):

389.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Magistral-Small-2509 at 15 t/s generation and 389 t/s prompt processing on an AMD Strix Halo 128GB system with a Radeon 8060S iGPU. Setup is llama.cpp via RADV with UD-Q4_K_XL, full offload, batch 2048 and flash attention. The figures are separate llama-bench pp512 and tg128 microbenchmarks, not one end-to-end request; AMDVLK gave 94 t/s prompt processing at the same 15 t/s generation.

generation:

40.0 tokens/s

prompt processing (prefill):

586.0 tokens/s

quant:

MXFP4 (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3-Coder-Next-80B at 40 t/s generation and 586 t/s prompt processing on a single Radeon 8060S iGPU in an AMD Strix Halo system with 128 GB unified memory. Setup is llama.cpp with the MXFP4 quant, full offload, batch 2048 and flash attention, using the RADV Vulkan backend. The figures are separate 512-token prompt and 128-token generation microbenchmarks, not one end-to-end request; the source notes post-hoc MXFP4 can underperform conventional quants.

generation:

56.0 tokens/s

prompt processing (prefill):

596.0 tokens/s

quant:

MXFP4 (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports GPT-OSS-120B at 56 t/s generation and 596 t/s prompt processing on an AMD Strix Halo 128GB system with a single Radeon 8060S iGPU. Setup is llama.cpp with MXFP4 and flash attention, full offload, batch 2048 and 32 threads. The figures are separate pp512 and tg128 microbenchmarks, not one end-to-end request; the same campaign lists AMDVLK at 661 t/s prompt and 53 t/s generation.