AMD Strix Halo 128GB reports

generation:

51.3 tokens/s

prompt processing (prefill):

2372.0 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next at 2,372 tok/s prefill on a 16K prompt and 51.3 tok/s decode at 8K context on Strix Halo 128GB. Setup is the hipfire 0.4.2.1 engine with MTP speculative decoding on, the XDNA2 NPU handling trunk input projections for two prefill banks alongside the iGPU. Prefill falls to 1,865 tok/s at 128K cold and 1,589 tok/s at 256K cold; decode is 41.8 tok/s at 32K, 42.1 at 128K and 35.6 at 256K. Numbers were measured on a 7.3-rc7 kernel with the IOMMU PerfOpt series, and the user expects lower prefill on stock kernels.

generation:

60.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next at 60 tok/s on a Strix Halo 128GB laptop (ASUS ProArt PX13, Ryzen AI Max+ 395, 96 GB set aside for GPU). Setup is Gufo (Thomas9120's Windows fork) with UD-Q4_K_XL quant and Q8_0 MTP draft model, 262144 context, speculative decoding with 7 draft tokens and prompt lookup enabled. Figures come from an 80-turn overnight coding run going from 45K to 262K context, not a controlled benchmark. User notes prompt lookup adds 20-30% on code-edit turns and the 10.16 draft changes add another 20-25%.

generation:

28.1 tokens/s

quant:

NVFP4 (NVFP4)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Nemotron 3.5 Super VL 120B-A12B at 28.1 tok/s on Strix Halo 128GB (Framework Desktop, Ryzen AI Max+ 395, gfx1151) under Linux with ROCm 7.13. Setup is Atlas Inference with NVFP4 weights and a LiLiCorr drafter (gamma 8) at 32K context with prefix caching on. Windows 11 gives 27.5 tok/s with the drafter and 18.4 tok/s without; Linux gives 17.8 tok/s without. Speculative decoding is lossless, byte-identical to plain decoding across all 995 BFCL prompts. BFCL ST-995 scores 88.14 and MLPerf Edge Agentic 0.611 across 1,007 recorded turns on both platforms. Text-only on AMD for now; vision runs on DGX Spark.

generation:

46.0 tokens/s

prompt processing (prefill):

818.1 tokens/s

quant:

MXFP4 (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports 46.05 t/s generation and 818.11 t/s prompt processing. Source evidence (verbatim): Source: https://nishtahir.com/gmktec-evo-x2-ryzen-ai-max-395-benchmarks/ # GMKtec EVO-X2 128GB: original GPT-OSS-120B MXFP4 ROCm llama-bench measurements Nish Tahir original test, October 23, 2025. GMKtec EVO-X2, Ryzen AI MAX+ 395 / Radeon 8060S gfx1151, 128GB unified memory (Strix Halo); lshw reports 128GiB. Ubuntu 24.04.3 LTS. Selected GPT-OSS-120B MXFP4 MoE GGUF llama-bench build 03792ad9 (6816), ROCm backend: pp2048 prefill 818.11 ±9.03 tokens/s; tg32 generation 46.05 ±0.18 tokens/s. Separate operation tests at zero existing context depth, not a combined 2048-token prompt followed by generation. Command settings: -fa 1 -d 0 -p 2048 -n 32 -ub 2048 --mmap 0; table ngl 99. Total parameters 116.83B; file 59.02GiB, not resident RAM. Core counts, repetition count, driver/ROCm package versions, KV precision, sampling, concurrent requests, resident memory and measured power unknown. Other depths/models/Ollama runs remain separate. Exact tested GGUF repository/revision unknown. Verified base-model reference only: https://huggingface.co/openai/gpt-oss-120b (official card: approximately117B total,5.1B active); not identification of tested GGUF. Preserve full sources and caveats in public Notes. Original: https://nishtahir.com/gmktec-evo-x2-ryzen-ai-max-395-benchmarks/ .

generation:

90.2 tokens/s

prompt processing (prefill):

1375.0 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next at 90.2 t/s decode and 1375 t/s prefill on a Strix Halo 128GB at 87k tokens of context. Setup is the strixite engine with cold, cache-defeated runs; the model is a hybrid recurrent/attention architecture whose KV cache is about 20 GiB at full depth. Decode decays from 94.6 t/s at 21k to 90.0 t/s at 261k, a 10% cost across 50x the memory. The user retracted earlier 765k t/s prefill figures as prefix-cache hits and reports the box runs 176.9B parameters at full working depth.

generation:

50.1 tokens/s

prompt processing (prefill):

442.1 tokens/s

quant:

Q4_K_M (GGUF)

kv:

Q8_0

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports 50.070235 t/s generation and 442.1505415 t/s prompt processing. Source evidence (verbatim): Source: https://www.soothill.io/blog/2026/08/19/vulkan-064-rocm-714-strix-halo/ # Strix Halo Qwen3-Coder Q4_K_M Vulkan measured 32K-depth benchmark First-hand Darren Soothill benchmark, measured/published 19 August 2026. Hardware GMKtec EVO-X3, Radeon 8060S gfx1151, 128GB unified memory (Strix Halo). CPU SKU and core counts not established by this article alone. Selected model Qwen3-Coder-30B-A3B-Q4_K_M, GGUF, llama.cpp Vulkan v0.6.4 build 10565 commit baf6360be (author short commit; full revision unknown). Selected measured context depth 32768 tokens already present before the operation; this is occupied depth, not allocated capacity. Microbatch 2048, batch2048, all available layers offloaded, Flash Attention enabled, Q8_0 K and V cache, one server slot, no-host loading. Native generation tg128 at that measured depth: 50.070235 tokens per second. Native prompt processing pp512 at the same depth and microbatch: 442.1505415 tokens per second. These are separate operation cells of the same matched synthetic configuration, not a single combined request or end-to-end rate. The author reports median of launch means. This 32K comparison used two launches per backend in ABBA order. Mesa26.3.0-devel RADV, libdrm2.4.134, shaderc2026.3-dev. IOMMU stayed enabled; no thermal, firmware or power-policy changes. Server slot count does not prove request concurrency; no concurrent request rate is inferred. Exact tested quant conversion repository, filename, checksum and revision are not published. Resident weight allocation and RAM consumption are unspecified. Original article https://www.soothill.io/blog/2026/08/19/vulkan-064-rocm-714-strix-halo/ . Original CSV https://www.soothill.io/assets/data/strix-halo-vulkan-064-rocm-714-2026-08-19.csv . Verified publisher model link https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct is a base-model reference, not the tested Q4_K_M artifact. Other backends, shallow cells, microbatches and served MTP requests are separate tests and must not be combined with the selected 32K synthetic cells.

generation:

67.5 tokens/s

quant:

Q4_K_S (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports 67.5288 t/s generation. Source evidence (verbatim): Source: https://www.soothill.io/blog/2026/08/14/coding-model-benchmark-strix-halo/ # Qwen3-Coder 30B-A3B Q4_K_S: first-hand EVO-X3 coding decode measurement Original author Darren Soothill measured this deployment on 14 August 2026, published 14 August and updated 15 August. Hardware: GMKtec EVO-X3, Ryzen AI MAX+ 395, Radeon 8060S, 128GB unified memory (Strix Halo). Selected native decode result: Qwen3-Coder-30B-A3B-Instruct-Q4_K_S, llama.cpp, 67.5288 tokens per second. The author's CSV records this separately from wall_seconds_per_task 3.9879 and completion_tokens_per_task 250.8. This is an aggregate deployment result across ten first-response coding tasks, not a standalone llama-bench tg128 test. The publication does not specify whether the decode aggregate is arithmetic or token-weighted; do not reinterpret its statistic. Runtime build/version, CPU/GPU core counts, OS, concurrency, actual per-task prompt tokens and prefill rate are not supplied. Every coding request used a configured 32768-token context capacity and 2048-token output ceiling, temperature 0, top-p 1, presence penalty 0, thinking disabled and no repair turn. Configured context capacity is not the actual input length or occupied context depth. The model has 30.5B total and 3.3B active parameters. CSV installed_gib 16.26 describes installed storage, not resident RAM or GPU allocation. The 5/10 task pass score is quality, not throughput. Other model/quant rows and their speeds are separate deployments and must not be combined with this selected row. Original article: https://www.soothill.io/blog/2026/08/14/coding-model-benchmark-strix-halo/ . Author's machine-readable source: https://www.soothill.io/assets/data/evox3-coding-model-benchmark-2026-08-14.csv . Verified Hugging Face link supplied by the author: https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct . This is a base-model reference, not the exact tested Q4_K_S quantization artifact. Tested conversion repository, filename checksum and revision are not established; never infer them from the base repository.

generation:

33.5 tokens/s

quant:

UD-Q4_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports 33.5 t/s generation. Source evidence (verbatim): Source: https://llmkube.com/docs/labs/qwen38-flash-next-gufo-on-strix-halo # First-hand Gufo Qwen3.8 Flash Next UD-Q4_K_XL single-stream generation on Strix Halo 128GB Christopher Maher's first-hand lab report, live deployment dated October 2, 2026. Measured AMD Ryzen AI Max+ 395 / Radeon 8060S gfx1151, Strix Halo 128GB. Installed unified memory: 128GB LPDDR5X. Selected single-stream agent-style run: generation 33.5 tokens/s, using MTP. Approximately 4K tokens of Go source plus coding instruction; up to 512 completion tokens; reasoning included, thinking defaults on. Server has two session slots, but this measured row has one stream, not two concurrent requests. Decode is completion tokens divided by generation time. Selected run prompt-processing speed unreported; leave promptTps null. Separate cold-prefill and two-stream rows must not be combined with this selected generation run. Qwen3.8-Flash-Next UD-Q4_K_XL, GGUF. Tested repository verified: https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF ; source reports short revision 38bb39ee, independently opened revision tree returned a generic tool error; full SHA not verified. Four quantization shards, 103.69 GiB on disk. Shared Q8_0 MTP head path: MTP/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf in that repository; 2.60 GiB disk size. Disk sizes are not resident RAM. Gufo HIP engine, context capacity 262144. Exact commit/version for selected bake-off row unestablished; later deployed Gufo v0.5.0/23cacbb9379d5a8e8531535270891729ca6c00dd is separate and must not be assigned to this earlier measurement. Core counts, power, selected resident memory, exact prompt count, CPU offload, KV quantization and sampling settings unreported. Lab author builds LLMKube. Client uses workstation kubectl port-forward to an otherwise idle node, one candidate at a time. Sources: https://llmkube.com/docs/labs/qwen38-flash-next-gufo-on-strix-halo ; https://llmkube.com/blog/qwen38-flash-next-gufo-strix-halo .

generation:

65.4 tokens/s

prompt processing (prefill):

861.0 tokens/s

quant:

MXFP4 (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports gpt-oss-20b at 65.4 t/s generation and 861 t/s prompt processing on an AMD Strix Halo 128GB system. Setup is native llama.cpp with the MXFP4 GGUF, flash attention on, mmap off and ngl 99, run through Vulkan (RADV GFX1151) with a 96 GB GPU carve-out of shared memory. Both figures come from llama-bench at depth 8192: PP512 at 860.95 t/s and a separate TG128 at 65.40 t/s, so they are not a same-request prompt/output pair.

generation:

6.4 tokens/s

prompt processing (prefill):

84.9 tokens/s

quant:

Q8_0 (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3-32B-Q8_0 at 6.4 t/s generation and 84.9 t/s prompt processing on an AMD Strix Halo 128GB system. Setup is llama.cpp with the Vulkan (RADV) backend, FA 1 and ngl 99, on a single AMD Radeon Graphics (RADV GFX1151) device. The figures come from separate llama-bench phases: PP512 at 84.87 +/- 0.35 t/s and TG128 at 6.43 +/- 0.01 t/s. Context depth, thread count and KV cache type are unreported.

generation:

6.4 tokens/s

prompt processing (prefill):

51.1 tokens/s

quant:

Q8_0 (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3-32B at 6.4 t/s generation and 51.1 t/s prompt processing on an AMD Strix Halo 128GB system. Setup is llama.cpp with Q8_0 and flash attention enabled, ngl 99, on one ROCm device (gfx1151). The figures are separate llama-bench phases: PP512 at 51.12 +/- 0.24 t/s and TG128 at 6.39 +/- 0.01 t/s, not a same-request pair. Context depth is not printed.

generation:

22.6 tokens/s

prompt processing (prefill):

247.2 tokens/s

quant:

UD-Q4_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.5 122B-A10B at 22.6 t/s generation and 247.2 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is llama-bench build b8171 with the Vulkan backend, UD-Q4_K_XL quant, ngl 99, and a Radeon 8060S Graphics (RADV GFX1151) device. The model is the 122B-A10B artifact at 63.65 GiB; PP512 is 512 benchmark prompt tokens and TG128 is 128 evaluation steps, so the two rates are not a same-request pair.

generation:

31.8 tokens/s

prompt processing (prefill):

747.7 tokens/s

quant:

UD-Q8_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.5 35B-A3B at 31.8 t/s generation and 747.7 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is llama-bench build b8171 with the Vulkan backend, UD-Q8_K_XL quant, ngl 99, and a Radeon 8060S Graphics (RADV GFX1151) device. The rates are separate synthetic phases: PP512 at 747.72 +/- 44.81 t/s and TG128 at 31.83 +/- 0.03 t/s, not a same-request prompt/output pair.

generation:

53.4 tokens/s

prompt processing (prefill):

865.7 tokens/s

quant:

UD-Q4_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.5 35B-A3B at 53.4 t/s generation and 865.7 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is llama-bench build b8171 with the Vulkan backend, UD-Q4_K_XL quant, ngl 99, and a Radeon 8060S Graphics (RADV GFX1151) device. The rates are separate synthetic phases: PP512 at 865.72 +/- 59.59 t/s and TG128 at 53.39 +/- 0.08 t/s, not a same-request prompt/output pair.

generation:

10.7 tokens/s

prompt processing (prefill):

220.4 tokens/s

quant:

UD-Q4_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.5 27B at 10.7 t/s generation and 220.4 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is the b8171 Vulkan build with the UD-Q4_K_XL GGUF, 26.90 B parameters and 16.40 GiB, ngl 99, on a Radeon 8060S (RADV GFX1151). The native figures are PP512 220.35 +/- 3.36 t/s and TG128 10.66 +/- 0.01 t/s, separate synthetic phases rather than a same-request pair; batch, threads, context depth and KV types are unreported.

generation:

31.1 tokens/s

prompt processing (prefill):

819.2 tokens/s

quant:

UD-Q4_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.5 9B at 31.1 t/s generation and 819.2 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is llama-bench build b8171 with the Vulkan backend, UD-Q4_K_XL quant, ngl 99, and a Radeon 8060S (RADV GFX1151) device. The rates are separate synthetic phases: PP512 at 819.24 +/- 55.72 t/s and TG128 at 31.09 +/- 0.05 t/s, not a same-request prompt/output pair.

generation:

28.0 tokens/s

prompt processing (prefill):

1259.1 tokens/s

quant:

UD-Q8_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.5-4B-UD-Q8_K_XL at 28 t/s generation and 1259.1 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is llama.cpp b8171 with the Vulkan backend, ngl 99, and the UD-Q8_K_XL quant; the native table lists type Q8_0, 4.21 B parameters and 5.53 GiB. The rates are separate synthetic phases, PP512 at 1259.14 +/- 3.82 t/s and TG128 at 27.95 +/- 0.07 t/s, not a same-request prompt/output pair.

generation:

48.5 tokens/s

prompt processing (prefill):

1388.9 tokens/s

quant:

UD-Q4_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.5-4B-UD-Q4_K_XL at 48.5 t/s generation and 1388.9 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is llama-bench build b8171 with the Vulkan backend, ngl 99, and the UD-Q4_K_XL quant, on a Radeon 8060S (RADV GFX1151) iGPU. The stored rates are rounded to 0.1 t/s from native PP512 1388.87 +/- 10.68 t/s and TG128 48.53 +/- 0.65 t/s, separate synthetic phases rather than a same-request pair.

generation:

50.2 tokens/s

prompt processing (prefill):

1493.3 tokens/s

quant:

Q4_0 (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Llama2 7B at 50.2 t/s generation and 1493.3 t/s prompt processing on an AMD Strix Halo 128GB system. Setup is llama.cpp with Q4_0, run natively on the Radeon 8060S iGPU via ROCm with -ngl 99 and flash attention; an attached RTX 5070 Ti does not participate. ROCm TG128 measured 50.16 +/- 0.07 t/s and PP512 1493.28 +/- 30.20 t/s; prompt processing fell to 958.19 t/s at 8192 tokens of context.

generation:

31.1 tokens/s

prompt processing (prefill):

104.8 tokens/s

quant:

UD-Q3_K_XL (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Minimax M2.1 at 31.1 t/s generation and 104.8 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is llama-bench on Vulkan0 with the UD-Q3_K_XL quant, -ngl 999, -p 256 -n 256, -t 16, -r 3 and -fa 1. The native figures are PP256 104.80 +/- 7.95 and TG256 31.13 +/- 0.02; a separate toolbox run gave PP256 104.50 +/- 8.57 and TG256 33.07 +/- 0.05.