Local model performance on your hardware

generation:

51.1 tokens/s

prompt processing (prefill):

173.5 tokens/s

quant:

MXFP4 (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Historical native llama-bench measurement published by GitHub user apellegr, measured 2026-03-10 UTC and audited 2026-10-13. Exact source snapshot c97e6cfb6f8503823cda8a791c6f6751ac52daf1. Selected model: gpt-oss-120b, GGUF MXFP4. Native mean PP512=173.496458 tokens/s (sample SD 0.286975); native mean TG128=51.118165 tokens/s (sample SD 0.021166). Three samples per phase. These are separate synthetic PP512 and TG128 tests at n_depth=0, not generation after a 512-token prompt and not HTTP end-to-end throughput. No TTFT or request latency is measured. Hardware: raw cpu_info identifies AMD Ryzen AI Max+ 395 with Radeon 8060S; raw gpu_info is exactly Radeon 8060S Graphics. Source guide reports 16 physical Zen 5 CPU cores, 40 GPU compute units, 128GB shared LPDDR5X-8000. Chassis, power mode, clocks, thermal state, run isolation and background load are unspecified. Participating GPU count is exactly 1: the run's pinned 05728db llama-bench get_gpu_info() iterates all registered GPU/IGPU devices and joins each description with commas, without deduplicating; this output contains one device. The same revision's default auto selection selects available GPUs, or iGPUs if no other GPU exists. Raw devices=auto, backends=ROCm, ngl=999, n_cpu_moe=0 and no tensor overrides support one-device GPU-offloaded execution. The guide additionally states HIP_VISIBLE_DEVICES=0. main_gpu=0 or tensor_split=0.00 alone would NOT prove count: the latter formatter trims trailing zeros. This establishes one participating GPU, not a census of unexposed installed hardware. Runtime conflict: every selected raw PP/TG row reports build_commit=05728db and build_number=1. GitHub resolves that revision to 05728db18eea59de81ee3a7699739daaf015206b. Use this raw build identity. The guide instead claims Lychee build b8280, commit b2e1427 for all March 10 benchmarks; that inconsistent claim is not assigned to these rows. The referenced Lychee repository returned 404 during audit. The actual fork/binary hash and any uncommitted changes remain unknown. Guide-reported surrounding stack is Ubuntu25.10, kernel6.17.0-14-generic, ROCm7.2.0, HIP7.2.26015.70200-43, rocBLAS5.2.0.70200-43, Clang22.0.0/GCC15.2.0 and gfx1151 target; these ancillary versions are not independently in raw rows. Raw settings: n_batch=512, n_ubatch=512, n_threads=16, K/V=f16, flash_attn=true, use_mmap=false, use_direct_io=false, no_kv_offload=false, split_mode=layer, n_gpu_layers=999, n_cpu_moe=0, embeddings=false, no_op_offload=0, no_host=false. PP n_prompt=512/n_gen=0; TG n_prompt=0/n_gen=128; both n_depth=0. Benchmark context request in pinned code is n_prompt+n_gen+n_depth, separately512/128 before runtime padding, not the model's supported context capacity. No long-context rate is implied. The guide's command specifies -r3; raw includes all3 samples. Pinned code performs default warmup before timed repetitions, clears KV state each repetition, uses synthetic/random tokens (BOS when appropriate), and synchronizes GPU execution. Exact full argv/no-warmup override is not logged. The elapsed timer is std::chrono::high_resolution_clock, with timestamps represented in nanoseconds and elapsed=end-start. Throughput is the arithmetic mean of per-sample token_count*1e9/elapsed_ns, verified from raw nanoseconds; it is not token_count divided by mean latency. This is a microbenchmark, not a language-quality or real-conversation test. PP timestamp 2026-03-10T20:31:39Z; elapsed ns samples [2956521834, 2949616216, 2947084768]. TG timestamp 2026-03-10T20:31:51Z; elapsed ns samples [2505184557, 2503577529, 2503245480]. Preserve raw rates rather than the guide's rounded values. The wrapper's quant=f16 conflicts with raw model_type=gpt-oss 120B MXFP4 MoE; f16 matches K/V dtype and must not be treated as weight quantization. Raw model_size=63374323968 is the runtime model/tensor payload byte count; wrapper model_size_gb=59.0 is a size label, not measured peak RAM. Full HF GGUF file bytes also include metadata and can differ from payload bytes. No artifact hash match is inferred from these sizes. Do not enter them as memory usage. System128GB is one shared pool, not128GB RAM plus128GB discrete VRAM. Guide GTT limit and approximate memory estimates are not per-run measured allocation; GPU reservation and peak memory remain unknown. Artifact provenance: raw basename gpt-oss-120b-mxfp4-00001-of-00003.gguf matches the pinned source downloader's ggml-org/gpt-oss-120b-GGUF destination and quant. The exact tested filename has three lowercase mxfp4 shards. The verified historical HF reference revision d932fcea62f83e088d8f076a2cd2d7eb02dfa682 predates the run. The repository history records deleting these shards on 2026-07-16; current main contains a different, single uppercase MXFP4 file. Use the historical three-shard link, never silently substitute current main. HF metadata verified exact file(s) at reference revision d932fcea62f83e088d8f076a2cd2d7eb02dfa682; this is an independently verified artifact reference, NOT a proven tested-weight revision/hash. The run supplies no HF revision, weight hash or download log. The exact measured publisher is attributed through the matching source downloader, not established from raw filename alone. Base/reference model is openai/gpt-oss-120b; it must not replace the GGUF artifact link. Only public text and metadata were read, with no model weights downloaded or source programs run. Sources: Raw: https://github.com/apellegr/Strix-Halo-Models/blob/c97e6cfb6f8503823cda8a791c6f6751ac52daf1/benchmarks/full_benchmark_20260310_183313.jsonl#L2993-L3080 Guide: https://github.com/apellegr/Strix-Halo-Models/blob/c97e6cfb6f8503823cda8a791c6f6751ac52daf1/docs/MODELS.md Downloader: https://github.com/apellegr/Strix-Halo-Models/blob/c97e6cfb6f8503823cda8a791c6f6751ac52daf1/scripts/setup/download_strix_halo_models.sh#L296-L302 Pinned GPU enumeration: https://github.com/ggml-org/llama.cpp/blob/05728db18eea59de81ee3a7699739daaf015206b/tools/llama-bench/llama-bench.cpp#L127-L136 Pinned device selection: https://github.com/ggml-org/llama.cpp/blob/05728db18eea59de81ee3a7699739daaf015206b/src/llama.cpp#L912-L979 Pinned timer: https://github.com/ggml-org/llama.cpp/blob/05728db18eea59de81ee3a7699739daaf015206b/tools/llama-bench/llama-bench.cpp#L35-L38 Pinned timing/metric code: https://github.com/ggml-org/llama.cpp/blob/05728db18eea59de81ee3a7699739daaf015206b/tools/llama-bench/llama-bench.cpp#L1378-L1392 and #L1962-L2012 and #L2159-L2258 Exact GGUF reference: https://huggingface.co/ggml-org/gpt-oss-120b-GGUF/blob/d932fcea62f83e088d8f076a2cd2d7eb02dfa682/gpt-oss-120b-mxfp4-00001-of-00003.gguf Full artifact tree: https://huggingface.co/ggml-org/gpt-oss-120b-GGUF/tree/d932fcea62f83e088d8f076a2cd2d7eb02dfa682 Base/reference: https://huggingface.co/openai/gpt-oss-120b Tensor payload size: https://github.com/ggml-org/llama.cpp/blob/05728db18eea59de81ee3a7699739daaf015206b/src/llama-model-loader.cpp#L555-L566 Structured PP/TG are rounded to the form’s 0.1 tokens/s step; exact source rates remain above.

generation:

69.0 tokens/s

prompt processing (prefill):

1538.2 tokens/s

quant:

Q6_K_L (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Historical native llama-bench measurement published by GitHub user apellegr, measured 2026-03-10 UTC and audited 2026-10-13. Exact source snapshot c97e6cfb6f8503823cda8a791c6f6751ac52daf1. Selected model: Llama-3.2-3B-Instruct, GGUF Q6_K_L. Native mean PP512=1538.174693 tokens/s (sample SD 4.022359); native mean TG128=69.042751 tokens/s (sample SD 0.017682). Three samples per phase. These are separate synthetic PP512 and TG128 tests at n_depth=0, not generation after a 512-token prompt and not HTTP end-to-end throughput. No TTFT or request latency is measured. Hardware: raw cpu_info identifies AMD Ryzen AI Max+ 395 with Radeon 8060S; raw gpu_info is exactly Radeon 8060S Graphics. Source guide reports 16 physical Zen 5 CPU cores, 40 GPU compute units, 128GB shared LPDDR5X-8000. Chassis, power mode, clocks, thermal state, run isolation and background load are unspecified. Participating GPU count is exactly 1: the run's pinned 05728db llama-bench get_gpu_info() iterates all registered GPU/IGPU devices and joins each description with commas, without deduplicating; this output contains one device. The same revision's default auto selection selects available GPUs, or iGPUs if no other GPU exists. Raw devices=auto, backends=ROCm, ngl=999, n_cpu_moe=0 and no tensor overrides support one-device GPU-offloaded execution. The guide additionally states HIP_VISIBLE_DEVICES=0. main_gpu=0 or tensor_split=0.00 alone would NOT prove count: the latter formatter trims trailing zeros. This establishes one participating GPU, not a census of unexposed installed hardware. Runtime conflict: every selected raw PP/TG row reports build_commit=05728db and build_number=1. GitHub resolves that revision to 05728db18eea59de81ee3a7699739daaf015206b. Use this raw build identity. The guide instead claims Lychee build b8280, commit b2e1427 for all March 10 benchmarks; that inconsistent claim is not assigned to these rows. The referenced Lychee repository returned 404 during audit. The actual fork/binary hash and any uncommitted changes remain unknown. Guide-reported surrounding stack is Ubuntu25.10, kernel6.17.0-14-generic, ROCm7.2.0, HIP7.2.26015.70200-43, rocBLAS5.2.0.70200-43, Clang22.0.0/GCC15.2.0 and gfx1151 target; these ancillary versions are not independently in raw rows. Raw settings: n_batch=2048, n_ubatch=512, n_threads=16, K/V=f16, flash_attn=true, use_mmap=false, use_direct_io=false, no_kv_offload=false, split_mode=layer, n_gpu_layers=999, n_cpu_moe=0, embeddings=false, no_op_offload=0, no_host=false. PP n_prompt=512/n_gen=0; TG n_prompt=0/n_gen=128; both n_depth=0. Benchmark context request in pinned code is n_prompt+n_gen+n_depth, separately512/128 before runtime padding, not the model's supported context capacity. No long-context rate is implied. The guide's command specifies -r3; raw includes all3 samples. Pinned code performs default warmup before timed repetitions, clears KV state each repetition, uses synthetic/random tokens (BOS when appropriate), and synchronizes GPU execution. Exact full argv/no-warmup override is not logged. The elapsed timer is std::chrono::high_resolution_clock, with timestamps represented in nanoseconds and elapsed=end-start. Throughput is the arithmetic mean of per-sample token_count*1e9/elapsed_ns, verified from raw nanoseconds; it is not token_count divided by mean latency. This is a microbenchmark, not a language-quality or real-conversation test. PP timestamp 2026-03-10T18:33:15Z; elapsed ns samples [333732087, 331991245, 332867406]. TG timestamp 2026-03-10T18:33:17Z; elapsed ns samples [1853572764, 1854464164, 1853734769]. Preserve raw rates rather than the guide's rounded values. The wrapper's quant=f16 conflicts with raw model_type=llama 3B Q6_K; f16 matches K/V dtype and must not be treated as weight quantization. Raw model_size=2731438336 is the runtime model/tensor payload byte count; wrapper model_size_gb=2.6 is a size label, not measured peak RAM. Full HF GGUF file bytes also include metadata and can differ from payload bytes. No artifact hash match is inferred from these sizes. Do not enter them as memory usage. System128GB is one shared pool, not128GB RAM plus128GB discrete VRAM. Guide GTT limit and approximate memory estimates are not per-run measured allocation; GPU reservation and peak memory remain unknown. Artifact provenance: raw basename Llama-3.2-3B-Instruct-Q6_K_L.gguf matches the pinned source downloader's bartowski/Llama-3.2-3B-Instruct-GGUF destination and quant. The exact artifact is Q6_K_L, not a plain Q6_K file. The runtime reports Q6_K; the bartowski card states Q6_K_L uses Q8_0 embedding/output tensors. Keep the Q6_K_L variant visible even if the catalog quant selector is Q6_K. HF metadata verified exact file(s) at reference revision 5ab33fa94d1d04e903623ae72c95d1696f09f9e8; this is an independently verified artifact reference, NOT a proven tested-weight revision/hash. The run supplies no HF revision, weight hash or download log. The exact measured publisher is attributed through the matching source downloader, not established from raw filename alone. Base/reference model is meta-llama/Llama-3.2-3B-Instruct; it must not replace the GGUF artifact link. Only public text and metadata were read, with no model weights downloaded or source programs run. Sources: Raw: https://github.com/apellegr/Strix-Halo-Models/blob/c97e6cfb6f8503823cda8a791c6f6751ac52daf1/benchmarks/full_benchmark_20260310_183313.jsonl#L1-L88 Guide: https://github.com/apellegr/Strix-Halo-Models/blob/c97e6cfb6f8503823cda8a791c6f6751ac52daf1/docs/MODELS.md Downloader: https://github.com/apellegr/Strix-Halo-Models/blob/c97e6cfb6f8503823cda8a791c6f6751ac52daf1/scripts/setup/download_strix_halo_models.sh#L172-L177 Pinned GPU enumeration: https://github.com/ggml-org/llama.cpp/blob/05728db18eea59de81ee3a7699739daaf015206b/tools/llama-bench/llama-bench.cpp#L127-L136 Pinned device selection: https://github.com/ggml-org/llama.cpp/blob/05728db18eea59de81ee3a7699739daaf015206b/src/llama.cpp#L912-L979 Pinned timer: https://github.com/ggml-org/llama.cpp/blob/05728db18eea59de81ee3a7699739daaf015206b/tools/llama-bench/llama-bench.cpp#L35-L38 Pinned timing/metric code: https://github.com/ggml-org/llama.cpp/blob/05728db18eea59de81ee3a7699739daaf015206b/tools/llama-bench/llama-bench.cpp#L1378-L1392 and #L1962-L2012 and #L2159-L2258 Exact GGUF reference: https://huggingface.co/bartowski/Llama-3.2-3B-Instruct-GGUF/blob/5ab33fa94d1d04e903623ae72c95d1696f09f9e8/Llama-3.2-3B-Instruct-Q6_K_L.gguf Full artifact tree: https://huggingface.co/bartowski/Llama-3.2-3B-Instruct-GGUF/tree/5ab33fa94d1d04e903623ae72c95d1696f09f9e8 Base/reference: https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct Tensor payload size: https://github.com/ggml-org/llama.cpp/blob/05728db18eea59de81ee3a7699739daaf015206b/src/llama-model-loader.cpp#L555-L566 Structured PP/TG are rounded to the form’s 0.1 tokens/s step; exact source rates remain above.

generation:

69.6 tokens/s

prompt processing (prefill):

812.0 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Historical native llama-bench measurement published by GitHub user apellegr, measured 2026-03-10 UTC and audited 2026-10-13. Exact source snapshot c97e6cfb6f8503823cda8a791c6f6751ac52daf1. Selected model: Qwen3-Coder-30B-A3B-Instruct, GGUF Q4_K_M. Native mean PP512=811.971025 tokens/s (sample SD 3.80521); native mean TG128=69.570608 tokens/s (sample SD 0.025147). Three samples per phase. These are separate synthetic PP512 and TG128 tests at n_depth=0, not generation after a 512-token prompt and not HTTP end-to-end throughput. No TTFT or request latency is measured. Hardware: raw cpu_info identifies AMD Ryzen AI Max+ 395 with Radeon 8060S; raw gpu_info is exactly Radeon 8060S Graphics. Source guide reports 16 physical Zen 5 CPU cores, 40 GPU compute units, 128GB shared LPDDR5X-8000. Chassis, power mode, clocks, thermal state, run isolation and background load are unspecified. Participating GPU count is exactly 1: the run's pinned 05728db llama-bench get_gpu_info() iterates all registered GPU/IGPU devices and joins each description with commas, without deduplicating; this output contains one device. The same revision's default auto selection selects available GPUs, or iGPUs if no other GPU exists. Raw devices=auto, backends=ROCm, ngl=999, n_cpu_moe=0 and no tensor overrides support one-device GPU-offloaded execution. The guide additionally states HIP_VISIBLE_DEVICES=0. main_gpu=0 or tensor_split=0.00 alone would NOT prove count: the latter formatter trims trailing zeros. This establishes one participating GPU, not a census of unexposed installed hardware. Runtime conflict: every selected raw PP/TG row reports build_commit=05728db and build_number=1. GitHub resolves that revision to 05728db18eea59de81ee3a7699739daaf015206b. Use this raw build identity. The guide instead claims Lychee build b8280, commit b2e1427 for all March 10 benchmarks; that inconsistent claim is not assigned to these rows. The referenced Lychee repository returned 404 during audit. The actual fork/binary hash and any uncommitted changes remain unknown. Guide-reported surrounding stack is Ubuntu25.10, kernel6.17.0-14-generic, ROCm7.2.0, HIP7.2.26015.70200-43, rocBLAS5.2.0.70200-43, Clang22.0.0/GCC15.2.0 and gfx1151 target; these ancillary versions are not independently in raw rows. Raw settings: n_batch=1024, n_ubatch=1024, n_threads=16, K/V=f16, flash_attn=true, use_mmap=false, use_direct_io=false, no_kv_offload=false, split_mode=layer, n_gpu_layers=999, n_cpu_moe=0, embeddings=false, no_op_offload=0, no_host=false. PP n_prompt=512/n_gen=0; TG n_prompt=0/n_gen=128; both n_depth=0. Benchmark context request in pinned code is n_prompt+n_gen+n_depth, separately512/128 before runtime padding, not the model's supported context capacity. No long-context rate is implied. The guide's command specifies -r3; raw includes all3 samples. Pinned code performs default warmup before timed repetitions, clears KV state each repetition, uses synthetic/random tokens (BOS when appropriate), and synchronizes GPU execution. Exact full argv/no-warmup override is not logged. The elapsed timer is std::chrono::high_resolution_clock, with timestamps represented in nanoseconds and elapsed=end-start. Throughput is the arithmetic mean of per-sample token_count*1e9/elapsed_ns, verified from raw nanoseconds; it is not token_count divided by mean latency. This is a microbenchmark, not a language-quality or real-conversation test. PP timestamp 2026-03-10T19:02:18Z; elapsed ns samples [632239336, 632310643, 627170762]. TG timestamp 2026-03-10T19:02:20Z; elapsed ns samples [1839249320, 1839755722, 1840567736]. Preserve raw rates rather than the guide's rounded values. The wrapper's quant=f16 conflicts with raw model_type=qwen3moe 30B.A3B Q4_K - Medium; f16 matches K/V dtype and must not be treated as weight quantization. Raw model_size=18550716416 is the runtime model/tensor payload byte count; wrapper model_size_gb=17.3 is a size label, not measured peak RAM. Full HF GGUF file bytes also include metadata and can differ from payload bytes. No artifact hash match is inferred from these sizes. Do not enter them as memory usage. System128GB is one shared pool, not128GB RAM plus128GB discrete VRAM. Guide GTT limit and approximate memory estimates are not per-run measured allocation; GPU reservation and peak memory remain unknown. Artifact provenance: raw basename Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf matches the pinned source downloader's unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF destination and quant. HF metadata verified exact file(s) at reference revision b17cb02dd882d5b6ab62fc777ad2995f19668350; this is an independently verified artifact reference, NOT a proven tested-weight revision/hash. The run supplies no HF revision, weight hash or download log. The exact measured publisher is attributed through the matching source downloader, not established from raw filename alone. Base/reference model is Qwen/Qwen3-Coder-30B-A3B-Instruct; it must not replace the GGUF artifact link. Only public text and metadata were read, with no model weights downloaded or source programs run. Sources: Raw: https://github.com/apellegr/Strix-Halo-Models/blob/c97e6cfb6f8503823cda8a791c6f6751ac52daf1/benchmarks/full_benchmark_20260310_183313.jsonl#L1233-L1320 Guide: https://github.com/apellegr/Strix-Halo-Models/blob/c97e6cfb6f8503823cda8a791c6f6751ac52daf1/docs/MODELS.md Downloader: https://github.com/apellegr/Strix-Halo-Models/blob/c97e6cfb6f8503823cda8a791c6f6751ac52daf1/scripts/setup/download_strix_halo_models.sh#L326-L330 Pinned GPU enumeration: https://github.com/ggml-org/llama.cpp/blob/05728db18eea59de81ee3a7699739daaf015206b/tools/llama-bench/llama-bench.cpp#L127-L136 Pinned device selection: https://github.com/ggml-org/llama.cpp/blob/05728db18eea59de81ee3a7699739daaf015206b/src/llama.cpp#L912-L979 Pinned timer: https://github.com/ggml-org/llama.cpp/blob/05728db18eea59de81ee3a7699739daaf015206b/tools/llama-bench/llama-bench.cpp#L35-L38 Pinned timing/metric code: https://github.com/ggml-org/llama.cpp/blob/05728db18eea59de81ee3a7699739daaf015206b/tools/llama-bench/llama-bench.cpp#L1378-L1392 and #L1962-L2012 and #L2159-L2258 Exact GGUF reference: https://huggingface.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF/blob/b17cb02dd882d5b6ab62fc777ad2995f19668350/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf Full artifact tree: https://huggingface.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF/tree/b17cb02dd882d5b6ab62fc777ad2995f19668350 Base/reference: https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct Tensor payload size: https://github.com/ggml-org/llama.cpp/blob/05728db18eea59de81ee3a7699739daaf015206b/src/llama-model-loader.cpp#L555-L566 Structured PP/TG are rounded to the form’s 0.1 tokens/s step; exact source rates remain above.

generation:

65.9 tokens/s

prompt processing (prefill):

401.5 tokens/s

quant:

UD-Q4_K_XL (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Historical first-party llama-bench measurements by robustonian (display name 金のニワトリ), published August 2025, inspected October 13, 2026. Source's EVO-X2 section updated August 3, 2025. Exact measurement timestamp is not published. This report covers only ROCm, Flash Attention on, on the EVO-X2. It does not combine the source's Mac Studio, CPU-only, RTX 3060 or RTX 3090 measurements. No local benchmark rerun. Hardware: EVO-X2 with AMD Ryzen AI Max+ 395, integrated Radeon 8060S / gfx1151, 128 GB unified installed memory. Source explicitly enumerates exactly one ROCm GPU in this benchmark invocation and the native output binds backend ROCm and ngl=100. This establishes one participating iGPU rather than inferring GPU quantity from one host or a model name. GPU core/CU count, physical CPU-core count, clock, TDP, thermal conditions and per-run memory usage are not measured in this log. 128 GB is installed unified RAM, not dedicated VRAM or resident model allocation. Native model size 16.45 GiB is the GGUF tensor/model footprint, not measured RAM use. Model: Qwen3-Coder-30B-A3B-Instruct, source-linked Unsloth GGUF artifact Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf. Preserve exact UD-Q4_K_XL identity; llama-bench's coarse model description is qwen3moe 30B.A3B Q4_K - Medium, 30.53 B total parameters. A3B is the family active-parameter label, not a measurement of activated weights. Current matching artifact metadata at HF revision b17cb02dd882d5b6ab62fc777ad2995f19668350: 17,665,334,432 bytes; LFS SHA256 2841aa314d916434860cfb8990347528dcdfe5c350dbcb9d1461dbee88ff2533. Current size is consistent with the source's rounded 16.45 GiB, but source did not publish a historical file hash or HF snapshot. This current reference is not claimed to be the tested historical bytes. No weights downloaded. Source-native rates, tokens/s (published value +/- printed deviation): headline pp512 401.48 +/- 2.72; separate tg128 65.93 +/- 0.18. Longer tests in the same backend/FA arm: pp8192 383.41 +/- 0.46; separate tg8192 63.27 +/- 0.17. These are separate native prefill and generation microbenchmarks, not an input prompt followed by a same-request answer, and tg8192 is not decode after a prefilled 8192-token prompt. Neither rate is derived from request time or TTFT. No TTFT is published or calculated. Native test labels identify executed fixed-count token work; these are synthetic benchmark token evaluations, not verified natural-language output tokens. Runtime: source explicitly labels ROCm 6.4.0, native ROCm backend, device0 AMD Radeon Graphics gfx1151 (0x1151), VMM no, wave size32; GGML_CUDA_FORCE_MMQ=no and GGML_CUDA_FORCE_CUBLAS=no are HIP/CUDA-compatibility diagnostic labels, not proof of a separate NVIDIA device. The shown command is llama-bench -m [exact named artifact] -ngl100 -fa0,1. Source omits the ROCm executable's llama.cpp build hash and build number. The long table does not repeat its command line or device enumeration; its same-machine/backend binding comes from continuity within the EVO-X2 ROCm6.4.0 section, rather than an independent second startup receipt. The Vulkan6060 or CUDA6068 hashes on this same page must not be inherited into this ROCm arm. Exact repetitions, batch/microbatch, KV dtype, effective context allocation, CPU threads, warmup and timing implementation therefore remain unverified for ROCm; the printed plus/minus values are retained without inventing their sample count. The native pp512/tg128/pp8192/tg8192 labels and phase rates are directly published, not independently reconstructed. Read this as a source-native result with incomplete runtime reproducibility. No quality test or natural-language output transcript is provided. The earlier introductory claim that HIP was untested is superseded by the explicitly labeled ROCm6.4.0 addendum, which contains these results. Sources: https://zenn.dev/robustonian/scraps/445f2143217f75#comment-ce5cc183b8f0cf https://huggingface.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF/blob/b17cb02dd882d5b6ab62fc777ad2995f19668350/Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct The form rounds the structured PP/TG rates to one decimal; the exact source values are retained above.

generation:

59.8 tokens/s

prompt processing (prefill):

409.5 tokens/s

quant:

UD-Q4_K_XL (GGUF)

flash attention:

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Historical first-party llama-bench measurements by robustonian (display name 金のニワトリ), published August 2025, inspected October 13, 2026. Source's EVO-X2 section updated August 3, 2025. Exact measurement timestamp is not published. This report covers only ROCm, Flash Attention off, on the EVO-X2. It does not combine the source's Mac Studio, CPU-only, RTX 3060 or RTX 3090 measurements. No local benchmark rerun. Hardware: EVO-X2 with AMD Ryzen AI Max+ 395, integrated Radeon 8060S / gfx1151, 128 GB unified installed memory. Source explicitly enumerates exactly one ROCm GPU in this benchmark invocation and the native output binds backend ROCm and ngl=100. This establishes one participating iGPU rather than inferring GPU quantity from one host or a model name. GPU core/CU count, physical CPU-core count, clock, TDP, thermal conditions and per-run memory usage are not measured in this log. 128 GB is installed unified RAM, not dedicated VRAM or resident model allocation. Native model size 16.45 GiB is the GGUF tensor/model footprint, not measured RAM use. Model: Qwen3-Coder-30B-A3B-Instruct, source-linked Unsloth GGUF artifact Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf. Preserve exact UD-Q4_K_XL identity; llama-bench's coarse model description is qwen3moe 30B.A3B Q4_K - Medium, 30.53 B total parameters. A3B is the family active-parameter label, not a measurement of activated weights. Current matching artifact metadata at HF revision b17cb02dd882d5b6ab62fc777ad2995f19668350: 17,665,334,432 bytes; LFS SHA256 2841aa314d916434860cfb8990347528dcdfe5c350dbcb9d1461dbee88ff2533. Current size is consistent with the source's rounded 16.45 GiB, but source did not publish a historical file hash or HF snapshot. This current reference is not claimed to be the tested historical bytes. No weights downloaded. Source-native rates, tokens/s (published value +/- printed deviation): headline pp512 409.48 +/- 0.81; separate tg128 59.78 +/- 0.40. Longer tests in the same backend/FA arm: pp8192 254.28 +/- 0.36; separate tg8192 30.78 +/- 0.08. These are separate native prefill and generation microbenchmarks, not an input prompt followed by a same-request answer, and tg8192 is not decode after a prefilled 8192-token prompt. Neither rate is derived from request time or TTFT. No TTFT is published or calculated. Native test labels identify executed fixed-count token work; these are synthetic benchmark token evaluations, not verified natural-language output tokens. Runtime: source explicitly labels ROCm 6.4.0, native ROCm backend, device0 AMD Radeon Graphics gfx1151 (0x1151), VMM no, wave size32; GGML_CUDA_FORCE_MMQ=no and GGML_CUDA_FORCE_CUBLAS=no are HIP/CUDA-compatibility diagnostic labels, not proof of a separate NVIDIA device. The shown command is llama-bench -m [exact named artifact] -ngl100 -fa0,1. Source omits the ROCm executable's llama.cpp build hash and build number. The long table does not repeat its command line or device enumeration; its same-machine/backend binding comes from continuity within the EVO-X2 ROCm6.4.0 section, rather than an independent second startup receipt. The Vulkan6060 or CUDA6068 hashes on this same page must not be inherited into this ROCm arm. Exact repetitions, batch/microbatch, KV dtype, effective context allocation, CPU threads, warmup and timing implementation therefore remain unverified for ROCm; the printed plus/minus values are retained without inventing their sample count. The native pp512/tg128/pp8192/tg8192 labels and phase rates are directly published, not independently reconstructed. Read this as a source-native result with incomplete runtime reproducibility. No quality test or natural-language output transcript is provided. The earlier introductory claim that HIP was untested is superseded by the explicitly labeled ROCm6.4.0 addendum, which contains these results. Sources: https://zenn.dev/robustonian/scraps/445f2143217f75#comment-ce5cc183b8f0cf https://huggingface.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF/blob/b17cb02dd882d5b6ab62fc777ad2995f19668350/Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct The form rounds the structured PP/TG rates to one decimal; the exact source values are retained above.

generation:

71.3 tokens/s

prompt processing (prefill):

399.8 tokens/s

quant:

UD-Q4_K_XL (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Historical first-party llama-bench measurements by robustonian (display name 金のニワトリ), published August 2025, inspected October 13, 2026. Source's EVO-X2 section updated August 3, 2025. Exact measurement timestamp is not published. This report covers only Vulkan, Flash Attention on, on the EVO-X2. It does not combine the source's Mac Studio, CPU-only, RTX 3060 or RTX 3090 measurements. No local benchmark rerun. Hardware: EVO-X2 with AMD Ryzen AI Max+ 395, integrated Radeon 8060S / gfx1151, 128 GB unified installed memory. Source explicitly enumerates exactly one Vulkan GPU in this benchmark invocation and the native output binds backend Vulkan and ngl=100. This establishes one participating iGPU rather than inferring GPU quantity from one host or a model name. GPU core/CU count, physical CPU-core count, clock, TDP, thermal conditions and per-run memory usage are not measured in this log. 128 GB is installed unified RAM, not dedicated VRAM or resident model allocation. Native model size 16.45 GiB is the GGUF tensor/model footprint, not measured RAM use. Model: Qwen3-Coder-30B-A3B-Instruct, source-linked Unsloth GGUF artifact Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf. Preserve exact UD-Q4_K_XL identity; llama-bench's coarse model description is qwen3moe 30B.A3B Q4_K - Medium, 30.53 B total parameters. A3B is the family active-parameter label, not a measurement of activated weights. Current matching artifact metadata at HF revision b17cb02dd882d5b6ab62fc777ad2995f19668350: 17,665,334,432 bytes; LFS SHA256 2841aa314d916434860cfb8990347528dcdfe5c350dbcb9d1461dbee88ff2533. Current size is consistent with the source's rounded 16.45 GiB, but source did not publish a historical file hash or HF snapshot. This current reference is not claimed to be the tested historical bytes. No weights downloaded. Source-native rates, tokens/s (mean +/- printed deviation): headline pp512 399.76 +/- 1.41; separate tg128 71.29 +/- 0.11. Longer tests in the same backend/FA arm: pp8192 315.36 +/- 0.33; separate tg8192 59.91 +/- 1.95. These are separate native prefill and generation microbenchmarks, not an input prompt followed by a same-request answer, and tg8192 is not decode after a prefilled 8192-token prompt. Neither rate is derived from request time or TTFT. No TTFT is published or calculated. Native test labels identify executed fixed-count token work; these are synthetic benchmark token evaluations, not verified natural-language output tokens. Runtime: llama.cpp build 6060, displayed 9c35706b9, resolved to 9c35706b98ea271858acef4194f526a71b24cdc9, Vulkan/RPC compiled backends. Driver identifies AMD Radeon Graphics (RADV GFX1151) (radv), uma=1, fp16=1, bf16=0, warp size64, shared memory65536, int dot1, matrix cores KHR_coopmat. Exact Mesa/RADV driver revision is unreported. RPC is compiled in, but no RPC server is requested by the shown commands and pinned default is empty; it does not indicate remote participation. Source short command uses -ngl100 -fa0,1; long command adds -p8192 -n8192. The long command repeats the one-device log, but not the build line; matching revision for that table is source-section continuity rather than an independent second build receipt. Read-only audit of pinned tools/llama-bench/llama-bench.cpp: defaults are five repetitions, depth0, requested batch2048/microbatch512, F16 K/V cache, main GPU0, mmap on, warmup enabled; source does not override them. Source does not print the effective CPU thread count in these Vulkan rows; CPU-only rows elsewhere are not substituted. PP and TG are generated as separate parameter instances. Context request is n_prompt+n_gen+n_depth:512 for pp512,128 for tg128,8192 for either longer phase. Effective allocation can be rounded by runtime; no 640-token combined request or262K context claim. Model/context setup and warmup lie outside the measured windows; KV is cleared before each repeat. Prompt warmup uses the full prompt count; generation warmup uses one token. test_prompt feeds random vocabulary IDs and synchronizes after all prompt batches; test_gen performs and synchronizes one llama_decode per step for exactly128/8192 steps, without EOS sampling. get_time_ns uses std::chrono::high_resolution_clock; get_ts uses fixed token count / elapsed nanoseconds, then arithmetic mean and sample standard deviation across the five repetitions. No first-token arrival measurement, sampler-quality gate, uncertainty beyond reported spread, or application-level latency is established. Sources: https://zenn.dev/robustonian/scraps/445f2143217f75 https://huggingface.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF/blob/b17cb02dd882d5b6ab62fc777ad2995f19668350/Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct https://github.com/ggml-org/llama.cpp/blob/9c35706b98ea271858acef4194f526a71b24cdc9/tools/llama-bench/llama-bench.cpp https://github.com/ggml-org/llama.cpp/commit/9c35706b98ea271858acef4194f526a71b24cdc9 The form rounds the structured PP/TG rates to one decimal; the exact source values are retained above.

generation:

73.0 tokens/s

prompt processing (prefill):

391.5 tokens/s

quant:

UD-Q4_K_XL (GGUF)

kv:

F16

flash attention:

off

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Historical first-party llama-bench measurements by robustonian (display name 金のニワトリ), published August 2025, inspected October 13, 2026. Source's EVO-X2 section updated August 3, 2025. Exact measurement timestamp is not published. This report covers only Vulkan, Flash Attention off, on the EVO-X2. It does not combine the source's Mac Studio, CPU-only, RTX 3060 or RTX 3090 measurements. No local benchmark rerun. Hardware: EVO-X2 with AMD Ryzen AI Max+ 395, integrated Radeon 8060S / gfx1151, 128 GB unified installed memory. Source explicitly enumerates exactly one Vulkan GPU in this benchmark invocation and the native output binds backend Vulkan and ngl=100. This establishes one participating iGPU rather than inferring GPU quantity from one host or a model name. GPU core/CU count, physical CPU-core count, clock, TDP, thermal conditions and per-run memory usage are not measured in this log. 128 GB is installed unified RAM, not dedicated VRAM or resident model allocation. Native model size 16.45 GiB is the GGUF tensor/model footprint, not measured RAM use. Model: Qwen3-Coder-30B-A3B-Instruct, source-linked Unsloth GGUF artifact Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf. Preserve exact UD-Q4_K_XL identity; llama-bench's coarse model description is qwen3moe 30B.A3B Q4_K - Medium, 30.53 B total parameters. A3B is the family active-parameter label, not a measurement of activated weights. Current matching artifact metadata at HF revision b17cb02dd882d5b6ab62fc777ad2995f19668350: 17,665,334,432 bytes; LFS SHA256 2841aa314d916434860cfb8990347528dcdfe5c350dbcb9d1461dbee88ff2533. Current size is consistent with the source's rounded 16.45 GiB, but source did not publish a historical file hash or HF snapshot. This current reference is not claimed to be the tested historical bytes. No weights downloaded. Source-native rates, tokens/s (mean +/- printed deviation): headline pp512 391.54 +/- 1.50; separate tg128 72.99 +/- 0.31. Longer tests in the same backend/FA arm: pp8192 284.71 +/- 0.45; separate tg8192 46.60 +/- 1.34. These are separate native prefill and generation microbenchmarks, not an input prompt followed by a same-request answer, and tg8192 is not decode after a prefilled 8192-token prompt. Neither rate is derived from request time or TTFT. No TTFT is published or calculated. Native test labels identify executed fixed-count token work; these are synthetic benchmark token evaluations, not verified natural-language output tokens. Runtime: llama.cpp build 6060, displayed 9c35706b9, resolved to 9c35706b98ea271858acef4194f526a71b24cdc9, Vulkan/RPC compiled backends. Driver identifies AMD Radeon Graphics (RADV GFX1151) (radv), uma=1, fp16=1, bf16=0, warp size64, shared memory65536, int dot1, matrix cores KHR_coopmat. Exact Mesa/RADV driver revision is unreported. RPC is compiled in, but no RPC server is requested by the shown commands and pinned default is empty; it does not indicate remote participation. Source short command uses -ngl100 -fa0,1; long command adds -p8192 -n8192. The long command repeats the one-device log, but not the build line; matching revision for that table is source-section continuity rather than an independent second build receipt. Read-only audit of pinned tools/llama-bench/llama-bench.cpp: defaults are five repetitions, depth0, requested batch2048/microbatch512, F16 K/V cache, main GPU0, mmap on, warmup enabled; source does not override them. Source does not print the effective CPU thread count in these Vulkan rows; CPU-only rows elsewhere are not substituted. PP and TG are generated as separate parameter instances. Context request is n_prompt+n_gen+n_depth:512 for pp512,128 for tg128,8192 for either longer phase. Effective allocation can be rounded by runtime; no 640-token combined request or262K context claim. Model/context setup and warmup lie outside the measured windows; KV is cleared before each repeat. Prompt warmup uses the full prompt count; generation warmup uses one token. test_prompt feeds random vocabulary IDs and synchronizes after all prompt batches; test_gen performs and synchronizes one llama_decode per step for exactly128/8192 steps, without EOS sampling. get_time_ns uses std::chrono::high_resolution_clock; get_ts uses fixed token count / elapsed nanoseconds, then arithmetic mean and sample standard deviation across the five repetitions. No first-token arrival measurement, sampler-quality gate, uncertainty beyond reported spread, or application-level latency is established. Sources: https://zenn.dev/robustonian/scraps/445f2143217f75 https://huggingface.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF/blob/b17cb02dd882d5b6ab62fc777ad2995f19668350/Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct https://github.com/ggml-org/llama.cpp/blob/9c35706b98ea271858acef4194f526a71b24cdc9/tools/llama-bench/llama-bench.cpp https://github.com/ggml-org/llama.cpp/commit/9c35706b98ea271858acef4194f526a71b24cdc9 The form rounds the structured PP/TG rates to one decimal; the exact source values are retained above.

generation:

12.2 tokens/s

prompt processing (prefill):

291.6 tokens/s

quant:

Q4_K_XL (GGUF)

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Primary raw source: https://github.com/ThinkOffApp/StrixLink/blob/5bc93e6c5ad136bc72f822be6c3ec95e78e67de8/docs/measurements-2026-09-10-raw.md Campaign description: https://github.com/ThinkOffApp/StrixLink/blob/5bc93e6c5ad136bc72f822be6c3ec95e78e67de8/README.md Author: ThinkOffApp. Source snapshot 5bc93e6c5ad136bc72f822be6c3ec95e78e67de8. Historical September 10–11, 2026 campaign, recovered October 13. Source names the September 10 rerun; raw log crosses UTC midnight. Native tables captured, not independently reproduced. Selected result: Qwen3.8-27B Q4_K_XL (author label; native Q4_K Medium), SOLO Bosgame M5. PP512 291.62 +/- 28.60 t/s and TG128 12.15 +/- 0.02 t/s, 5 repeats, arithmetic mean/sample SD according to exact native runtime. These are separate synthetic tests, not a512-input/128-output client request; no RPC split or speculative-server rate is imported. Hardware: Bosgame M5, AMD Ryzen AI Max+395 / Radeon8060S, nominal128GB unified RAM, Ubuntu26.04 as stated for the rig. CPU/GPU physical core counts, native thread count, exact September kernel/driver versions and active power limit unknown. GPU count 1 is source-attributed at same-host/campaign scope: raw table labels 'M5 alone, Vulkan ngl99', native backend Vulkan with noRPC; the same file's later00:37:55Z DeepSeek section says the Strix-alone row was already recorded and skipped, followed by bench-tail output 'Found 1 Vulkan devices' and device0 Radeon8060S RADV. This is explicit same-host/campaign enumeration, not a distinct per-selected-process count log. Source also describes two machines' GPUs(Metal+Vulkan). No count inferred merely from one device name. Memory configuration: September nominal1GB BIOS carve and120GB GTT addressable region; these are configuration limits, not measured model-resident memory. Native tensor size 16.34GiB; not diskbytes or residency. The raw 'gtt after' value is post-benchmark and is not a peak/resident model measurement. Native parameter count 27.32B is rounded. Model/artifact: Author names Q4_K_XL; native type displays qwen35 27B Q4_K - Medium. Preserve both; quant publisher and exact filename unpublished. Balanced power profile explicitly attributed in summary; August used accelerator-performance, so no memory-carve causal comparison. Tested GGUF revision, SHA256 and quant publisher unknown. HF link below is only the verified canonical base-model card, NOT the measured quant artifact. It cannot establish historical tested bytes. HF reference only: https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/README.md Runtime: campaign README explicitly binds every September10 row to llama.cpp434ddbb, Vulkan+RPC build on Strix but selected execution Vulkan only; the later split row also has build434ddbb(345). Per-selected-row binary hashes/footers unavailable. Exact434ddbb llama-bench source inspected, no source software run. Core native timer excludes model loading/context creation; times synchronized batch prefill or sequential single-token decode; numerator n_prompt+n_gen. PP512 andTG128 labels contain no @d suffix, consistent with depth0 in exact formatter. Full commands unavailable, so batch/KV/FA/warmup/configuration values are unknown rather than silently inherited from defaults. Model-loaded performance does not include cold-load or RPC upload. Measured TTFT unknown; no reciprocal rate or other proxy supplied. Requested context would be512/128 for separate zero-depth arms in this exact timer; effective allocated capacity not logged. No quality/correctness/generalization claim based on the separate sky-blue smoke. Timer reference: https://github.com/ggml-org/llama.cpp/blob/434ddbb/tools/llama-bench/llama-bench.cpp Caveats: source narrative gives a same-campaign runtime rather than process-local footer; soleGPU proof is same-campaign rather than perprocess. Different Mac-only and RPC arms, August superseded runs, live-server/speculative rates, transport throughput and load durations are excluded. LlamaPerf numeric fields are rounded to the supported 0.1 tok/s step; exact source means and standard deviations are retained above.

generation:

18.6 tokens/s

prompt processing (prefill):

140.3 tokens/s

quant:

IQ3_XXS (GGUF)

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Primary raw source: https://github.com/ThinkOffApp/StrixLink/blob/5bc93e6c5ad136bc72f822be6c3ec95e78e67de8/docs/measurements-2026-09-10-raw.md Campaign description: https://github.com/ThinkOffApp/StrixLink/blob/5bc93e6c5ad136bc72f822be6c3ec95e78e67de8/README.md Author: ThinkOffApp. Source snapshot 5bc93e6c5ad136bc72f822be6c3ec95e78e67de8. Historical September 10–11, 2026 campaign, recovered October 13. Source names the September 10 rerun; raw log crosses UTC midnight. Native tables captured, not independently reproduced. Selected result: DeepSeek V4 Flash IQ3_XXS, SOLO Bosgame M5. PP512 140.30 +/- 4.35 t/s and TG128 18.56 +/- 0.01 t/s, 3 repeats, arithmetic mean/sample SD according to exact native runtime. These are separate synthetic tests, not a512-input/128-output client request; no RPC split or speculative-server rate is imported. Hardware: Bosgame M5, AMD Ryzen AI Max+395 / Radeon8060S, nominal128GB unified RAM, Ubuntu26.04 as stated for the rig. CPU/GPU physical core counts, native thread count, exact September kernel/driver versions and active power limit unknown. GPU count 1 is source-attributed at same-host/campaign scope: raw table labels 'M5 alone, Vulkan ngl99', native backend Vulkan with noRPC; the same file's later00:37:55Z DeepSeek section says the Strix-alone row was already recorded and skipped, followed by bench-tail output 'Found 1 Vulkan devices' and device0 Radeon8060S RADV. This is explicit same-host/campaign enumeration, not a distinct per-selected-process count log. Source also describes two machines' GPUs(Metal+Vulkan). No count inferred merely from one device name. Memory configuration: September nominal1GB BIOS carve and120GB GTT addressable region; these are configuration limits, not measured model-resident memory. Native tensor size 97.05GiB; not diskbytes or residency. The raw 'gtt after' value is post-benchmark and is not a peak/resident model measurement. Native parameter count 284.33B is rounded. Model/artifact: Direct IO (dio) explicitly shown in both native rows. Four shards stated; filenames and exact disk bytes unpublished. Tested GGUF revision, SHA256 and quant publisher unknown. HF link below is only the verified canonical base-model card, NOT the measured quant artifact. It cannot establish historical tested bytes. HF reference only: https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash/blob/60d8d70770c6776ff598c94bb586a859a38244f1/README.md Runtime: campaign README explicitly binds every September10 row to llama.cpp434ddbb, Vulkan+RPC build on Strix but selected execution Vulkan only; the later split row also has build434ddbb(345). Per-selected-row binary hashes/footers unavailable. Exact434ddbb llama-bench source inspected, no source software run. Core native timer excludes model loading/context creation; times synchronized batch prefill or sequential single-token decode; numerator n_prompt+n_gen. PP512 andTG128 labels contain no @d suffix, consistent with depth0 in exact formatter. Full commands unavailable, so batch/KV/FA/warmup/configuration values are unknown rather than silently inherited from defaults. Model-loaded performance does not include cold-load or RPC upload. Measured TTFT unknown; no reciprocal rate or other proxy supplied. Requested context would be512/128 for separate zero-depth arms in this exact timer; effective allocated capacity not logged. No quality/correctness/generalization claim based on the separate sky-blue smoke. Timer reference: https://github.com/ggml-org/llama.cpp/blob/434ddbb/tools/llama-bench/llama-bench.cpp Caveats: source narrative gives a same-campaign runtime rather than process-local footer; soleGPU proof is same-campaign rather than perprocess. Different Mac-only and RPC arms, August superseded runs, live-server/speculative rates, transport throughput and load durations are excluded. LlamaPerf numeric fields are rounded to the supported 0.1 tok/s step; exact source means and standard deviations are retained above.

generation:

13.2 tokens/s

prompt processing (prefill):

370.9 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Source: https://github.com/hogeheer499-commits/strix-halo-guide/blob/f5d9e6dac1ec9aeeeeb1165ae85d305b33cee234/data/raw/2026-09-26/64gb-tier-30b-candidates-b11146/README.md Author: hogeheer499-commits. Historical measurement 2026-09-26, recovered 2026-10-13; source snapshot f5d9e6dac1ec9aeeeeb1165ae85d305b33cee234. Raw CSV and JSON, per-run commands and stderr retained. This is routine-load first-party benchmark evidence, not independent reproduction. Hardware: Beelink GTR9 Pro, AMD Ryzen AI Max+ 395 / Radeon 8060S, nominal 128GB unified RAM. Each of the four measured processes explicitly enumerates exactly one Vulkan device, Radeon 8060S; native rows say Vulkan. RPC library being loaded is not RPC participation. Native CPU threads16. Separate May1 hardware audit says16 physical cores/32 threads; these are historical same-host topology, not an active-core measurement in September. GPU CU count unknown. No dedicated VRAM or resident/peak model memory measurement. Runtime: official Ubuntu x64 Vulkan llama.cpp v0.5.0 release asset, executable reports0.5.0-dev build11146 commit7fe450e19, GNU11.4.0. Kernel7.0.0-31, Mesa/RADV26.2.3, AMD_VULKAN_ICD=RADV. All rows use ngl999, FA1, batch2048, ubatch512, F16 K/V, load/lazy auto, devices:auto, main_gpu0, layer split, no CPU MoE override. Host: balanced power, DPMauto, VM/desktop/browser/video-call/remote-access processes active; Ollama empty; preexisting swap nearly full. No strict-clean or causal cross-campaign comparison. Model: Muse Glimmer 30B, Q4_K_M; publisher meta-models; Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf. GGUF text file is dense with 27.85B-class parameters, despite marketed 30B name. Text-only; no mmproj, vision, DFlash or speculative decoding. Artifact: pinned HF 70bf1b61ac09f91b24d39038091b41c582bc5d7a; 16756683904 disk bytes; SHA256 4cc57c0f51040a226e5a72cc47b7613f7772950e460a665f7083de89f183f60e. Source's local hash and byte count match independently retrieved pinned HF metadata. Actual tested bytes remain author-attested; no weight download or independent rehash. Native model_size=16743567360 is tensor storage, not disk file size or residency. HF: https://huggingface.co/meta-models/Muse-Glimmer-30B-GGUF/blob/70bf1b61ac09f91b24d39038091b41c582bc5d7a/Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf Native rows (mean +/- sample SD; pass2 is a distinct repeated control, not pooled or best-of): - Pass 1 pp512, prepared depth 0, r10: 370.886965 +/- 1.316720 t/s; mean native duration 1380490185 ns. - Pass 1 tg128, prepared depth 0, r10: 13.239989 +/- 0.009645 t/s; mean native duration 9667686709 ns. - Pass 2 pp512, prepared depth 0, r10: 369.722541 +/- 0.872760 t/s; mean native duration 1384829181 ns. - Pass 2 tg128, prepared depth 0, r10: 13.241908 +/- 0.006578 t/s; mean native duration 9666282889 ns. - Pass 1 pp8192, prepared depth 0, r3: 346.515729 +/- 0.886785 t/s; mean native duration 23641165577 ns. - Pass 1 tg128, prepared depth 8192, r3: 13.014330 +/- 0.022881 t/s; mean native duration 9835332625 ns. The headline pair is pass1 PP512/TG128. Separate synthetic tests, not a512-input/128-output chat request. Exact commit's llama-bench timer inspected: load, context creation, warmup and depth preparation precede timing; PP processes random token batches and synchronizes; TG does128 single-token decode calls with synchronization. Numerator is n_prompt+n_gen, not useful natural-language output. The depth8192 TG test preloads/restores8192 tokens outside timing, then measures128 new decode steps. PP8192 starts at depth0. Requested n_ctx is512/128/8192/8320 per corresponding shape; effective allocated capacity is not logged and stays unknown. Native durations are not TTFT. Measured TTFT unknown; no reciprocal-rate or prompt/time proxy supplied. No server/client rate, concurrency, speculation or quality inference imported. Timer: https://github.com/ggml-org/llama.cpp/blob/7fe450e19/tools/llama-bench/llama-bench.cpp LlamaPerf numeric fields are rounded to the supported 0.1 tok/s step; exact source means and standard deviations are retained above.

generation:

80.4 tokens/s

prompt processing (prefill):

1399.0 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Source: https://github.com/hogeheer499-commits/strix-halo-guide/blob/f5d9e6dac1ec9aeeeeb1165ae85d305b33cee234/data/raw/2026-09-26/64gb-tier-30b-candidates-b11146/README.md Author: hogeheer499-commits. Historical measurement 2026-09-26, recovered 2026-10-13; source snapshot f5d9e6dac1ec9aeeeeb1165ae85d305b33cee234. Raw CSV and JSON, per-run commands and stderr retained. This is routine-load first-party benchmark evidence, not independent reproduction. Hardware: Beelink GTR9 Pro, AMD Ryzen AI Max+ 395 / Radeon 8060S, nominal 128GB unified RAM. Each of the four measured processes explicitly enumerates exactly one Vulkan device, Radeon 8060S; native rows say Vulkan. RPC library being loaded is not RPC participation. Native CPU threads16. Separate May1 hardware audit says16 physical cores/32 threads; these are historical same-host topology, not an active-core measurement in September. GPU CU count unknown. No dedicated VRAM or resident/peak model memory measurement. Runtime: official Ubuntu x64 Vulkan llama.cpp v0.5.0 release asset, executable reports0.5.0-dev build11146 commit7fe450e19, GNU11.4.0. Kernel7.0.0-31, Mesa/RADV26.2.3, AMD_VULKAN_ICD=RADV. All rows use ngl999, FA1, batch2048, ubatch512, F16 K/V, load/lazy auto, devices:auto, main_gpu0, layer split, no CPU MoE override. Host: balanced power, DPMauto, VM/desktop/browser/video-call/remote-access processes active; Ollama empty; preexisting swap nearly full. No strict-clean or causal cross-campaign comparison. Model: Laguna XS 2.1, Q4_K_M; publisher ggml-org; Laguna-XS-2.1-Q4_K_M.gguf. GGUF native type is laguna 30B.A3B Q4_K - Medium, 33.442617088B parameters. Separate code smoke exhausted 1024-token cap and passed with n4096; that is not this synthetic workload. Artifact: pinned HF 273068c9ae4ee6efea803da861aa404d115d3023; 19563570240 disk bytes; SHA256 0a8301bc1b8509b27ed39ec86b98278c03341db6311c065326bd6629cd7304d6. Source's local hash and byte count match independently retrieved pinned HF metadata. Actual tested bytes remain author-attested; no weight download or independent rehash. Native model_size=19559844864 is tensor storage, not disk file size or residency. HF: https://huggingface.co/ggml-org/Laguna-XS-2.1-GGUF/blob/273068c9ae4ee6efea803da861aa404d115d3023/Laguna-XS-2.1-Q4_K_M.gguf Native rows (mean +/- sample SD; pass2 is a distinct repeated control, not pooled or best-of): - Pass 1 pp512, prepared depth 0, r10: 1399.017430 +/- 9.822885 t/s; mean native duration 365987368 ns. - Pass 1 tg128, prepared depth 0, r10: 80.446103 +/- 0.116181 t/s; mean native duration 1591130400 ns. - Pass 2 pp512, prepared depth 0, r10: 1385.834889 +/- 9.886393 t/s; mean native duration 369469348 ns. - Pass 2 tg128, prepared depth 0, r10: 80.312458 +/- 0.540978 t/s; mean native duration 1593841291 ns. - Pass 1 pp8192, prepared depth 0, r3: 1026.839530 +/- 0.519353 t/s; mean native duration 7977878875 ns. - Pass 1 tg128, prepared depth 8192, r3: 70.052156 +/- 0.037432 t/s; mean native duration 1827210360 ns. The headline pair is pass1 PP512/TG128. Separate synthetic tests, not a512-input/128-output chat request. Exact commit's llama-bench timer inspected: load, context creation, warmup and depth preparation precede timing; PP processes random token batches and synchronizes; TG does128 single-token decode calls with synchronization. Numerator is n_prompt+n_gen, not useful natural-language output. The depth8192 TG test preloads/restores8192 tokens outside timing, then measures128 new decode steps. PP8192 starts at depth0. Requested n_ctx is512/128/8192/8320 per corresponding shape; effective allocated capacity is not logged and stays unknown. Native durations are not TTFT. Measured TTFT unknown; no reciprocal-rate or prompt/time proxy supplied. No server/client rate, concurrency, speculation or quality inference imported. Timer: https://github.com/ggml-org/llama.cpp/blob/7fe450e19/tools/llama-bench/llama-bench.cpp LlamaPerf numeric fields are rounded to the supported 0.1 tok/s step; exact source means and standard deviations are retained above.

generation:

66.8 tokens/s

prompt processing (prefill):

1450.3 tokens/s

quant:

Q4_0 (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Source: https://github.com/hogeheer499-commits/strix-halo-guide/blob/f5d9e6dac1ec9aeeeeb1165ae85d305b33cee234/data/raw/2026-09-26/64gb-tier-30b-candidates-b11146/README.md Author: hogeheer499-commits. Historical measurement 2026-09-26, recovered 2026-10-13; source snapshot f5d9e6dac1ec9aeeeeb1165ae85d305b33cee234. Raw CSV and JSON, per-run commands and stderr retained. This is routine-load first-party benchmark evidence, not independent reproduction. Hardware: Beelink GTR9 Pro, AMD Ryzen AI Max+ 395 / Radeon 8060S, nominal 128GB unified RAM. Each of the four measured processes explicitly enumerates exactly one Vulkan device, Radeon 8060S; native rows say Vulkan. RPC library being loaded is not RPC participation. Native CPU threads16. Separate May1 hardware audit says16 physical cores/32 threads; these are historical same-host topology, not an active-core measurement in September. GPU CU count unknown. No dedicated VRAM or resident/peak model memory measurement. Runtime: official Ubuntu x64 Vulkan llama.cpp v0.5.0 release asset, executable reports0.5.0-dev build11146 commit7fe450e19, GNU11.4.0. Kernel7.0.0-31, Mesa/RADV26.2.3, AMD_VULKAN_ICD=RADV. All rows use ngl999, FA1, batch2048, ubatch512, F16 K/V, load/lazy auto, devices:auto, main_gpu0, layer split, no CPU MoE override. Host: balanced power, DPMauto, VM/desktop/browser/video-call/remote-access processes active; Ollama empty; preexisting swap nearly full. No strict-clean or causal cross-campaign comparison. Model: NVIDIA Nemotron 3.5 Lightning 30B-A3B, Q4_0; publisher ggml-org; NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Q4_0.gguf. GGUF native type is nemotron_h_moe 31B.A3.5B Q4_0; metadata parameter count 31.577940288B differs from the rounded marketed 30B-A3B name. No MTP sidecar or speculative decoding. Artifact: pinned HF 8a08a1c81dadcc75d35dbb96016cfd344b632e67; 18898091584 disk bytes; SHA256 61f87e75974e4b535dcdf9aad056541a9514f1dfa4538b463b081d19b7a00e3c. Source's local hash and byte count match independently retrieved pinned HF metadata. Actual tested bytes remain author-attested; no weight download or independent rehash. Native model_size=18890195712 is tensor storage, not disk file size or residency. HF: https://huggingface.co/ggml-org/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF/blob/8a08a1c81dadcc75d35dbb96016cfd344b632e67/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Q4_0.gguf Native rows (mean +/- sample SD; pass2 is a distinct repeated control, not pooled or best-of): - Pass 1 pp512, prepared depth 0, r10: 1450.331991 +/- 3.306742 t/s; mean native duration 353024272 ns. - Pass 1 tg128, prepared depth 0, r10: 66.801892 +/- 0.140252 t/s; mean native duration 1916120989 ns. - Pass 2 pp512, prepared depth 0, r10: 1446.360644 +/- 6.329379 t/s; mean native duration 353998024 ns. - Pass 2 tg128, prepared depth 0, r10: 67.147905 +/- 0.082188 t/s; mean native duration 1906242227 ns. - Pass 1 pp8192, prepared depth 0, r3: 1409.099000 +/- 2.766984 t/s; mean native duration 5813658993 ns. - Pass 1 tg128, prepared depth 8192, r3: 65.987632 +/- 0.078190 t/s; mean native duration 1939759257 ns. The headline pair is pass1 PP512/TG128. Separate synthetic tests, not a512-input/128-output chat request. Exact commit's llama-bench timer inspected: load, context creation, warmup and depth preparation precede timing; PP processes random token batches and synchronizes; TG does128 single-token decode calls with synchronization. Numerator is n_prompt+n_gen, not useful natural-language output. The depth8192 TG test preloads/restores8192 tokens outside timing, then measures128 new decode steps. PP8192 starts at depth0. Requested n_ctx is512/128/8192/8320 per corresponding shape; effective allocated capacity is not logged and stays unknown. Native durations are not TTFT. Measured TTFT unknown; no reciprocal-rate or prompt/time proxy supplied. No server/client rate, concurrency, speculation or quality inference imported. Timer: https://github.com/ggml-org/llama.cpp/blob/7fe450e19/tools/llama-bench/llama-bench.cpp LlamaPerf numeric fields are rounded to the supported 0.1 tok/s step; exact source means and standard deviations are retained above.

generation:

54.2 tokens/s

prompt processing (prefill):

1049.7 tokens/s

quant:

Q8_0 (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

First-party source: https://howquant.dev/benchmarks/strix-halo-rocm-vs-vulkan/ (published 2026-08-24, updated 2026-09-16; native measurement date not explicitly given). This is historical source recovery, not a new run. The author says raw JSON is retained privately, not publicly downloadable; these are published summary cells, not independently inspected raw runs. Model Qwen3.6-35B-A3B-MTP Q8_0, filename Qwen3.6-35B-A3B-MTP-Q8_0.gguf, 35.5B parameters. Author-reported disk file size 37,790,106,112 bytes is not resident memory. CRITICAL HF provenance: Historical provenance is unresolved: author attributes this MTP-named GGUF to bartowski, while the verified same-name reference is published by ggml-org. Reference has 37,801,096,544 bytes versus author-reported tested file 37,790,106,112 bytes. Do not substitute ggml-org as tested publisher or claim matching bytes. Model/quant remain source-reported Qwen3.6-35B-A3B-MTP Q8_0. Historical revision and hash unknown. Verified pinned reference: https://huggingface.co/ggml-org/Qwen3.6-35B-A3B-MTP-GGUF/blob/f2da312dab4999371e5b3d00de96ae5f951f1d85/Qwen3.6-35B-A3B-MTP-Q8_0.gguf . No weights downloaded. Hardware: Corsair AI Workstation 300, AMD Ryzen AI Max+ 395 / Radeon 8060S gfx1151, 128 GiB unified RAM (source explicitly GiB; not dedicated VRAM), Balanced power mode. Ubuntu 24.04.4 LTS, kernel 6.18.36-061836-generic. Source reports memory 8000 MT/s configured/8532 measured; method for measured speed unspecified. Official SKU specifications are 16 CPU cores, 32 hardware threads and 40 graphics CUs, not measured active counts: https://www.amd.com/en/products/processors/laptop/ryzen/ai-300-series/amd-ryzen-ai-max-plus-395.html . Native benchmark thread count unknown; do not borrow server --threads 14. GPU quantity 1 is bound to Vulkan by https://howquant.dev/guides/vulkan-compute-strix-halo/#runtime-device-selection : author explicitly identifies one real GPU plus CPU software llvmpipe, retained native JSON devices: auto, and run attributed to Radeon 8060S. This is author-reported campaign evidence; raw JSON is not public. Same author's eGPU experiments are distinct and are not added to this run. ROCm companion guide shows sole ROCm0 on the machine but not independently per-run count; HIP cells are retained as companion evidence only. Qualified headline and all Vulkan values, tok/s: pp512 1049.70 ±7.40; pp2048 973.60 ±17.50; pp8192 911.00 ±7.90; tg128 54.22 ±0.09; pg2048+256 330.20 ±1.30. Same-backend PP512/TG128 headline; never pair HIP prefill with Vulkan generation. Reported SD 0.00 is rounded, not proof of zero variability. Source-reported HIP companion rates (GPU count not independently bound to each run): pp512 1091.00 ±48.20; pp2048 1010.70 ±16.70; pp8192 925.30 ±4.70; tg128 47.47 ±0.07; pg2048+256 307.40 ±0.40. Runtime llama.cpp c060ca974, build 10603; Vulkan RADV/Mesa 26.1.7. Exact native commands use -ngl 999 -b 2048 -ub 512 -fa on -r 5; plain decoding, no speculation even if filename says MTP. Native PP sizes 512/2048/8192 are timed synthetic work, not maximum context capacity. TG128 is 128 sequential synchronized random-token decode work units, not delivered text. Exact timer checked: https://github.com/ggml-org/llama.cpp/blob/c060ca974/tools/llama-bench/llama-bench.cpp . Per-repetition rate=(n_prompt+n_gen)/elapsed; mean and sample SD across 5. Model load/warmup precede timer; KV cleared before each repetition; no pre-existing depth. Separate requested contexts 512/2048/8192 for PP and 128 for TG; PG requests 2304 and times 2048+256 together. Effective allocated capacity unknown. PG rate is not separate PP, TG or client E2E. F16 K/V and enabled warmup derive from exact command and historical defaults. No measured TTFT, first-content latency, resident memory, model-load latency or client throughput is submitted. SPEED-Bench coding/MTP values and its configured 131072 context are a separate client/server workload with different timing; excluded from these native fields. No native long-context claim is taken from that server capacity. LlamaPerf structured rates are rounded to 0.1 tokens/s to match form precision; exact source means and SDs are retained above.

generation:

7.8 tokens/s

prompt processing (prefill):

278.8 tokens/s

quant:

Q8_0 (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

First-party source: https://howquant.dev/benchmarks/strix-halo-rocm-vs-vulkan/ (published 2026-08-24, updated 2026-09-16; native measurement date not explicitly given). This is historical source recovery, not a new run. The author says raw JSON is retained privately, not publicly downloadable; these are published summary cells, not independently inspected raw runs. Model Qwen3.8-27B Q8_0, filename Qwen3.8-27B-Q8_0.gguf, 27.3B parameters. Author-reported disk file size 29,105,393,664 bytes is not resident memory. CRITICAL HF provenance: The pinned bartowski reference matches publisher, model, quant and filename, but has 29,116,388,960 bytes versus the author-reported tested file 29,105,393,664 bytes. Therefore this is not verified historical byte identity; the model card reports later reuploads. Historical revision and hash unknown. Verified pinned reference: https://huggingface.co/bartowski/Qwen3.8-27B-GGUF/blob/44e6f088cbf95fffb83ec2564f30e63b26cd187e/Qwen3.8-27B-Q8_0.gguf . No weights downloaded. Hardware: Corsair AI Workstation 300, AMD Ryzen AI Max+ 395 / Radeon 8060S gfx1151, 128 GiB unified RAM (source explicitly GiB; not dedicated VRAM), Balanced power mode. Ubuntu 24.04.4 LTS, kernel 6.18.36-061836-generic. Source reports memory 8000 MT/s configured/8532 measured; method for measured speed unspecified. Official SKU specifications are 16 CPU cores, 32 hardware threads and 40 graphics CUs, not measured active counts: https://www.amd.com/en/products/processors/laptop/ryzen/ai-300-series/amd-ryzen-ai-max-plus-395.html . Native benchmark thread count unknown; do not borrow server --threads 14. GPU quantity 1 is bound to Vulkan by https://howquant.dev/guides/vulkan-compute-strix-halo/#runtime-device-selection : author explicitly identifies one real GPU plus CPU software llvmpipe, retained native JSON devices: auto, and run attributed to Radeon 8060S. This is author-reported campaign evidence; raw JSON is not public. Same author's eGPU experiments are distinct and are not added to this run. ROCm companion guide shows sole ROCm0 on the machine but not independently per-run count; HIP cells are retained as companion evidence only. Qualified headline and all Vulkan values, tok/s: pp512 278.80 ±9.40; pp2048 268.30 ±0.20; pp8192 254.20 ±0.10; tg128 7.77 ±0.00; pg2048+256 55.18 ±0.01. Same-backend PP512/TG128 headline; never pair HIP prefill with Vulkan generation. Reported SD 0.00 is rounded, not proof of zero variability. Source-reported HIP companion rates (GPU count not independently bound to each run): pp512 341.30 ±12.40; pp2048 322.80 ±0.60; pp8192 303.60 ±0.20; tg128 7.76 ±0.00; pg2048+256 58.30 ±0.03. Runtime llama.cpp c060ca974, build 10603; Vulkan RADV/Mesa 26.1.7. Exact native commands use -ngl 999 -b 2048 -ub 512 -fa on -r 5; plain decoding, no speculation even if filename says MTP. Native PP sizes 512/2048/8192 are timed synthetic work, not maximum context capacity. TG128 is 128 sequential synchronized random-token decode work units, not delivered text. Exact timer checked: https://github.com/ggml-org/llama.cpp/blob/c060ca974/tools/llama-bench/llama-bench.cpp . Per-repetition rate=(n_prompt+n_gen)/elapsed; mean and sample SD across 5. Model load/warmup precede timer; KV cleared before each repetition; no pre-existing depth. Separate requested contexts 512/2048/8192 for PP and 128 for TG; PG requests 2304 and times 2048+256 together. Effective allocated capacity unknown. PG rate is not separate PP, TG or client E2E. F16 K/V and enabled warmup derive from exact command and historical defaults. No measured TTFT, first-content latency, resident memory, model-load latency or client throughput is submitted. SPEED-Bench coding/MTP values and its configured 131072 context are a separate client/server workload with different timing; excluded from these native fields. No native long-context claim is taken from that server capacity. LlamaPerf structured rates are rounded to 0.1 tokens/s to match form precision; exact source means and SDs are retained above.

generation:

85.1 tokens/s

prompt processing (prefill):

1917.9 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Primary result: https://github.com/bkpaine1/halo-llamacpp/blob/3617c72a325d03f73da0764d6b64d386e2db1a9c/bench/ollama-headtohead-llama3.2-3b.md Pinned campaign README: https://github.com/bkpaine1/halo-llamacpp/blob/3617c72a325d03f73da0764d6b64d386e2db1a9c/README.md Model: source llama3.2:3b Q4_K_M. Exact tested Ollama GGUF blob SHA256 supplied by source: dde5aa3fc5ffc17176b5e8bdc82f587b24b2678c6c66101bf7da77af9f7ccdff. Public current metadata https://ollama.com/library/llama3.2:3b/blobs/dde5aa3fc5ff verifies the matching short blob prefix, llama architecture and Q4_K_M. We did not download/re-hash weights; full hash remains author-reported. The same source reports direct llama-bench PP5121917.92/TG12885.07 tok/s; this is a first-party textual native summary, not the complete raw table, and no SD/per-trial output is preserved. Requested n_ctx512/128 under the documented d0/default convention; effective allocation unknown. Source also gives Ollama prompt-eval880.53 on38 actual prompt tokens and eval82.08, but Ollama version, output count and comparable timing/workload are missing. Those companion values are excluded from structured native metrics; do not call the comparison apples-to-apples or claim a causal3.6% runtime gain. No PP/TG cross-pairing. First-party bkpaine1 measurements dated 2026-07-23. Source repository https://github.com/bkpaine1/halo-llamacpp, capture revision 3617c72a325d03f73da0764d6b64d386e2db1a9c. Historical qualification of previously deferred evidence; no new measurement or source-change claim. Hardware: source README identifies AMD Ryzen AI Max+ 395, Radeon 8060S gfx1151 and 128 GB unified memory. Its explicit statement that every run enumerates one ROCm device, with Device 0 AMD Radeon Graphics/gfx1151, binds GPU quantity 1 to this native campaign. This is a common campaign-level device snippet, not a repeated device block in each result file. Vendor SKU specification confirms 16 CPU cores, 32 hardware threads and 40 graphics compute units: https://www.amd.com/en/products/processors/laptop/ryzen/ai-300-series/amd-ryzen-ai-max-plus-395.html . Actual benchmark thread count is not reported. Host label AI-Strix; manufacturer, machine model, distribution, kernel, memory speed and tested power setting unknown. The enumerated 65536 MiB VRAM is driver-reported GPU-addressable capacity in the unified pool, not 64 GiB additional physical memory or measured residency. Model/KV/process resident memory and exact disk bytes unknown. Runtime: llama.cpp commit 1425386fd996511e1f3295e7366c38289a92a271, ROCm/HIP 7.2.4 (hipconfig 7.2.53211), ROCm clang-22. Source common command: ROCBLAS_USE_HIPBLASLT=1 llama-bench -m <model> -ngl 999 -fa 1; build enables HIP, gfx1151, rocWMMA FATTN and HIP_NO_VMM. The raw tables report build 1425386 (1), ROCm, ngl999 and fa1. Build number 1 is reported literally, not a modern release tag. No speculative drafter in these native runs. Live services shared this working box, so no isolated hardware or backend causal comparison is established. Exact historical timer source: https://github.com/ggml-org/llama.cpp/blob/1425386fd996511e1f3295e7366c38289a92a271/tools/llama-bench/llama-bench.cpp . PP512 runs 512 synthetic random-token decode work units, then synchronizes. TG128 runs 128 synchronized single-token decode work units; these are not client-visible generated text tokens. Rates are per-trial (n_prompt+n_gen)/elapsed, then arithmetic mean and sample SD where supplied. Depth is prepared/restored before timing and excluded from both rate numerator and denominator. TTFT, client latency, request wall time and model-load time are not measured. No rate-derived TTFT proxy is supplied. The exact source defaults are batch2048, ubatch512, f16 K/V, five repetitions and warmup enabled; full per-run command lines are absent, so defaults are labeled source defaults rather than independently observed settings. bench.sh uses --no-mmap while the common README command omits it; load mode is left unknown rather than grafted from that recipe. Effective allocated context capacity is not reported. HF link https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct is a canonical parent-model/architecture reference ONLY, verified public metadata revision 0cb88a4f764b7a12671c53f0838cd831a0843b95. It is not the tested GGUF or evidence of that quant publisher; no reference-model bytes are claimed to match the benchmark. No weights downloaded. LlamaPerf numeric fields are rounded to the supported 0.1 tok/s step; exact source rates and standard deviations are retained above.

generation:

73.0 tokens/s

prompt processing (prefill):

961.2 tokens/s

quant:

MXFP4 (GGUF)

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Primary result: https://github.com/bkpaine1/halo-llamacpp/blob/3617c72a325d03f73da0764d6b64d386e2db1a9c/bench/gpt-oss-20b.md Pinned campaign README: https://github.com/bkpaine1/halo-llamacpp/blob/3617c72a325d03f73da0764d6b64d386e2db1a9c/README.md Model: gpt-oss-20b, native gpt-oss 20B MXFP4 MoE,20.91B parameters. Tensor-storage size11.27 GiB is not disk-file bytes or resident memory. Tested GGUF publisher, filename, revision and hash are unknown. Observed PP512 961.24 ±390.50 and TG128 73.01 ±0.10 tok/s. CRITICAL: author explicitly attributes the enormous PP variability to GPU contention from live services; preserve the PP as a noisy observed measurement, not a representative uncontended speed. TG is described by the author as more reliable, but still comes from the contended working machine. No confidence interval or independent repeats beyond the printed summary are supplied. Requested n_ctx512 for PP and128 for TG; effective allocation unknown. First-party bkpaine1 measurements dated 2026-07-23. Source repository https://github.com/bkpaine1/halo-llamacpp, capture revision 3617c72a325d03f73da0764d6b64d386e2db1a9c. Historical qualification of previously deferred evidence; no new measurement or source-change claim. Hardware: source README identifies AMD Ryzen AI Max+ 395, Radeon 8060S gfx1151 and 128 GB unified memory. Its explicit statement that every run enumerates one ROCm device, with Device 0 AMD Radeon Graphics/gfx1151, binds GPU quantity 1 to this native campaign. This is a common campaign-level device snippet, not a repeated device block in each result file. Vendor SKU specification confirms 16 CPU cores, 32 hardware threads and 40 graphics compute units: https://www.amd.com/en/products/processors/laptop/ryzen/ai-300-series/amd-ryzen-ai-max-plus-395.html . Actual benchmark thread count is not reported. Host label AI-Strix; manufacturer, machine model, distribution, kernel, memory speed and tested power setting unknown. The enumerated 65536 MiB VRAM is driver-reported GPU-addressable capacity in the unified pool, not 64 GiB additional physical memory or measured residency. Model/KV/process resident memory and exact disk bytes unknown. Runtime: llama.cpp commit 1425386fd996511e1f3295e7366c38289a92a271, ROCm/HIP 7.2.4 (hipconfig 7.2.53211), ROCm clang-22. Source common command: ROCBLAS_USE_HIPBLASLT=1 llama-bench -m <model> -ngl 999 -fa 1; build enables HIP, gfx1151, rocWMMA FATTN and HIP_NO_VMM. The raw tables report build 1425386 (1), ROCm, ngl999 and fa1. Build number 1 is reported literally, not a modern release tag. No speculative drafter in these native runs. Live services shared this working box, so no isolated hardware or backend causal comparison is established. Exact historical timer source: https://github.com/ggml-org/llama.cpp/blob/1425386fd996511e1f3295e7366c38289a92a271/tools/llama-bench/llama-bench.cpp . PP512 runs 512 synthetic random-token decode work units, then synchronizes. TG128 runs 128 synchronized single-token decode work units; these are not client-visible generated text tokens. Rates are per-trial (n_prompt+n_gen)/elapsed, then arithmetic mean and sample SD where supplied. Depth is prepared/restored before timing and excluded from both rate numerator and denominator. TTFT, client latency, request wall time and model-load time are not measured. No rate-derived TTFT proxy is supplied. The exact source defaults are batch2048, ubatch512, f16 K/V, five repetitions and warmup enabled; full per-run command lines are absent, so defaults are labeled source defaults rather than independently observed settings. bench.sh uses --no-mmap while the common README command omits it; load mode is left unknown rather than grafted from that recipe. Effective allocated context capacity is not reported. HF link https://huggingface.co/openai/gpt-oss-20b is a canonical parent-model/architecture reference ONLY, verified public metadata revision 6cee5e81ee83917806bbde320786a8fb61efebee. It is not the tested GGUF or evidence of that quant publisher; no reference-model bytes are claimed to match the benchmark. No weights downloaded. LlamaPerf numeric fields are rounded to the supported 0.1 tok/s step; exact source rates and standard deviations are retained above.

generation:

73.7 tokens/s

prompt processing (prefill):

843.3 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Primary result: https://github.com/bkpaine1/halo-llamacpp/blob/3617c72a325d03f73da0764d6b64d386e2db1a9c/bench/qwen3-30b-a3b.md Pinned campaign README: https://github.com/bkpaine1/halo-llamacpp/blob/3617c72a325d03f73da0764d6b64d386e2db1a9c/README.md Model identity is the README Qwen3-30B-A3B-Instruct-2507 plus raw qwen3moe 30B.A3B Q4_K - Medium, 30.53B parameters. Native model tensor-storage size 17.28 GiB comes from llama_model_size; it is not exact disk bytes or resident memory. The commented Qwen3-30B GGUF filename in bench.sh is a TODO example, not measured publisher identity. Tested GGUF filename, publisher, repository revision and hash are unknown. Headline uses the same d0 ROCm arm: PP843.25 ±5.39 and TG73.74 ±0.17 tok/s. At d4096, PP209.17 ±0.17 and TG47.48 ±0.08. Depth4096 is pre-existing KV state, not a 4096-token timed prefill. Requested n_ctx equals depth+work:512/128 at d0 and4608/4224 at d4096; allocated capacity unknown. First-party bkpaine1 measurements dated 2026-07-23. Source repository https://github.com/bkpaine1/halo-llamacpp, capture revision 3617c72a325d03f73da0764d6b64d386e2db1a9c. Historical qualification of previously deferred evidence; no new measurement or source-change claim. Hardware: source README identifies AMD Ryzen AI Max+ 395, Radeon 8060S gfx1151 and 128 GB unified memory. Its explicit statement that every run enumerates one ROCm device, with Device 0 AMD Radeon Graphics/gfx1151, binds GPU quantity 1 to this native campaign. This is a common campaign-level device snippet, not a repeated device block in each result file. Vendor SKU specification confirms 16 CPU cores, 32 hardware threads and 40 graphics compute units: https://www.amd.com/en/products/processors/laptop/ryzen/ai-300-series/amd-ryzen-ai-max-plus-395.html . Actual benchmark thread count is not reported. Host label AI-Strix; manufacturer, machine model, distribution, kernel, memory speed and tested power setting unknown. The enumerated 65536 MiB VRAM is driver-reported GPU-addressable capacity in the unified pool, not 64 GiB additional physical memory or measured residency. Model/KV/process resident memory and exact disk bytes unknown. Runtime: llama.cpp commit 1425386fd996511e1f3295e7366c38289a92a271, ROCm/HIP 7.2.4 (hipconfig 7.2.53211), ROCm clang-22. Source common command: ROCBLAS_USE_HIPBLASLT=1 llama-bench -m <model> -ngl 999 -fa 1; build enables HIP, gfx1151, rocWMMA FATTN and HIP_NO_VMM. The raw tables report build 1425386 (1), ROCm, ngl999 and fa1. Build number 1 is reported literally, not a modern release tag. No speculative drafter in these native runs. Live services shared this working box, so no isolated hardware or backend causal comparison is established. Exact historical timer source: https://github.com/ggml-org/llama.cpp/blob/1425386fd996511e1f3295e7366c38289a92a271/tools/llama-bench/llama-bench.cpp . PP512 runs 512 synthetic random-token decode work units, then synchronizes. TG128 runs 128 synchronized single-token decode work units; these are not client-visible generated text tokens. Rates are per-trial (n_prompt+n_gen)/elapsed, then arithmetic mean and sample SD where supplied. Depth is prepared/restored before timing and excluded from both rate numerator and denominator. TTFT, client latency, request wall time and model-load time are not measured. No rate-derived TTFT proxy is supplied. The exact source defaults are batch2048, ubatch512, f16 K/V, five repetitions and warmup enabled; full per-run command lines are absent, so defaults are labeled source defaults rather than independently observed settings. bench.sh uses --no-mmap while the common README command omits it; load mode is left unknown rather than grafted from that recipe. Effective allocated context capacity is not reported. HF link https://huggingface.co/Qwen/Qwen3-30B-A3B-Instruct-2507 is a canonical parent-model/architecture reference ONLY, verified public metadata revision 0d7cf23991f47feeb3a57ecb4c9cee8ea4a17bfe. It is not the tested GGUF or evidence of that quant publisher; no reference-model bytes are claimed to match the benchmark. No weights downloaded. LlamaPerf numeric fields are rounded to the supported 0.1 tok/s step; exact source rates and standard deviations are retained above.

generation:

12.1 tokens/s

prompt processing (prefill):

334.0 tokens/s

quant:

Q4_K_M (GGUF)

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Qwen3.6-27B-MTP Q4_K_M native PP/TG Headline is native raw PP512 333.95±7.08 / TG128 12.05±0.03, plus PP64 240.26±22.23 / TG64 12.00±0.05. The method states 3 repetitions; raw logs confirm FA on, mmap off and requested ngl 999. This is raw inference without speculative decoding. The separate CLI MTP attempt hung with 0% GPU and was killed; it supplies no measured result. A third-party 20 tokens/s claim and source's implied 1.67x speculation estimate are excluded. Tested local filename Qwen3.6-27B-MTP-Q4_K_M.gguf is visible in the failed b867 log and adjacent method; successful model label has 27.32 B parameters and 15.92 GiB tensor size. Source says downloads were from unsloth. Current reference repo unsloth/Qwen3.6-27B-MTP-GGUF contains Qwen3.6-27B-Q4_K_M.gguf, lacking -MTP in filename. Thus HF is a model/quantization reference only, not an exact-filename or byte-identity match. Do not claim that current file hash was tested. Exact measured engine short hash 39cf5d6 is in both successful logs; matching upstream timer source was read. Source calls runtime lemonade b1270, while raw build counter is 1; preserve both instead of substituting upstream release numbers. Historical source-reported native llama-bench measurements by Paul Durkin / NorthstarAurora / h34v3nzc0dex. Evidence dataset revision 4c24348922a9f2ff7e481956c0eb08fa28284227; guide snapshot 46d062142eae35e36f0c1d0d91b393b59724432a. These are evidence snapshots, not engine or tested-model revisions. Exact measurement date is unpublished; the dataset existed by 2026-05-25. No new benchmark was run. Corsair AI Workstation 300, Ryzen AI MAX+ 395 / Radeon 8060S gfx1151, nominal 128 GB unified LPDDR5X-8000. Every retained raw log explicitly enumerates found 1 ROCm/Vulkan device and device 0, then shows that backend and positive GPU-layer offload. Thus participating physical GPU quantity = 1 per arm. Alternate ROCm/Vulkan builds run on the same APU, not two GPUs. Source also labels memory 128 GiB; retain nominal 128 GB SKU and source units, without claiming measured physical byte capacity. 131072 MiB reported ROCm VRAM is unified exposed capacity, not dedicated VRAM or resident weight memory. CPU thread count, native resident memory, measured CPU/GPU core counts and concurrency are not established by these logs. PP and TG are separate fixed-work synthetic tests, not one chat request. PP64/PP512 token counts are actual llama_decode input work; TG64/TG128 makes 64/128 single-token decode calls, without EOS-driven text sampling. These logs have no nonzero depth suffix; native test labels indicate depth 0. No 2048-token or deep-context test is attributed to this 27B campaign. Native timers use std::chrono high_resolution_clock nanoseconds. Load/context setup, warmup, per-repetition memory clearing precede timing. The implementation can populate depth before timing, but these 27B tests have depth 0. Timed functions include host token construction and llama_decode; PP synchronizes after its batches and TG synchronizes each token. Per-repetition rate is 1e9*(n_prompt+n_gen)/elapsed_ns; reported values are mean and sample SD of rates over 3 stated repetitions. The source initializes demand as n_prompt+n_gen+n_depth, but may reuse a larger context; actual allocation request and rounded capacity are unknown. No tokenization, HTTP/client timing or measured TTFT is provided; TTFT remains blank. No prompt_tokens/rate proxy is used. Native batch/KV settings are not in compact logs. Checked engine source defaults are batch 2048, ubatch 512, F16 KV, but defaults are not recorded run fields and remain reference-only. Requested offload 99/999 is not a verified number of model layers. HF links identify current reference metadata. Historical tested revision and hash remain unknown. Rounded native model tensor-storage size is neither disk file bytes nor measured resident memory. No weights were downloaded; no source software was executed. Raw rows (tokens/s mean±SD; d denotes preinitialized depth): qwen36-bench/bench-qwen36-27b-mtp-raw-pp512-tg128-lemonade-b1270.log FA 1 pp512 333.95±7.08; FA 1 tg128 12.05±0.03 qwen36-bench/bench-qwen36-27b-mtp-raw-pp64-tg64-lemonade-b1270.log FA 1 pp64 240.26±22.23; FA 1 tg64 12.0±0.05 Sources: https://huggingface.co/datasets/NorthstarAurora/strix-halo-bench-data/blob/4c24348922a9f2ff7e481956c0eb08fa28284227/qwen36-bench/bench-qwen36-27b-mtp-raw-pp512-tg128-lemonade-b1270.log https://huggingface.co/datasets/NorthstarAurora/strix-halo-bench-data/blob/4c24348922a9f2ff7e481956c0eb08fa28284227/qwen36-bench/bench-qwen36-27b-mtp-raw-pp64-tg64-lemonade-b1270.log https://github.com/h34v3nzc0dex/strix-halo-llm-finetune-guide/blob/46d062142eae35e36f0c1d0d91b393b59724432a/qwen36-bench/README.md https://huggingface.co/unsloth/Qwen3.6-27B-MTP-GGUF/blob/5cb35eb3dcbf52dbce5f87dbc64df6aaffadcace/Qwen3.6-27B-Q4_K_M.gguf Source QA: Compact logs start with device enumeration, not the command, despite the dataset card saying the first non-comment line is the command. Command shapes and repetitions come from the linked campaign guide and bench script. Raw build hashes are distinguished from guide release labels. No Zen 5c claim is copied from the dataset card. Timer source matching reported engine hash: https://github.com/ggml-org/llama.cpp/blob/39cf5d6/tools/llama-bench/llama-bench.cpp Hardware source: https://huggingface.co/datasets/NorthstarAurora/strix-halo-bench-data/blob/4c24348922a9f2ff7e481956c0eb08fa28284227/README.md LlamaPerf structured rates are rounded to 0.1 tokens/s to match form precision; exact native means and SDs are retained above.

generation:

21.5 tokens/s

prompt processing (prefill):

484.0 tokens/s

quant:

BF16 (GGUF)

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Qwen3.6-35B-A3B BF16 native PP/TG Headline is ROCm FA=1 shape 1, PP512 484.01±4.40 and TG128 21.51±0.00. Separate shape 2 at depth 0 yields PP2048 474.69±2.76 and TG128 23.71±0.01. Do not mix shape 1 prefill with shape 2 generation. Vulkan shape 1 is PP512 305.21±0.87 / TG128 10.74±0.00; its depth-sweep TG128 at d0 is 10.73±0.00. Zero SD is the source's rounded display, not proof of noiseless timing. Both BF16 backends report a497476 (1), matched by source method b9296. Source reproduction names unsloth/Qwen3.6-35B-A3B-GGUF and first shard BF16/Qwen3.6-35B-A3B-BF16-00001-of-00002.gguf; current pinned reference includes both shards. Native tensor size 64.60 GiB and params 34.66 B are not measured resident memory. Do not claim Vulkan native BF16 compute from BF16 weight storage: measured capability explicitly reports bf16:0. Historical source-reported native llama-bench measurements by Paul Durkin / NorthstarAurora / h34v3nzc0dex. Evidence dataset revision 4c24348922a9f2ff7e481956c0eb08fa28284227; guide snapshot 46d062142eae35e36f0c1d0d91b393b59724432a. These are evidence snapshots, not engine or tested-model revisions. Exact measurement date is unpublished; the dataset existed by 2026-05-25. No new benchmark was run. Corsair AI Workstation 300, Ryzen AI MAX+ 395 / Radeon 8060S gfx1151, nominal 128 GB unified LPDDR5X-8000. Every retained raw log explicitly enumerates found 1 ROCm/Vulkan device and device 0, then shows that backend and positive GPU-layer offload. Thus participating physical GPU quantity = 1 per arm. Alternate ROCm/Vulkan builds run on the same APU, not two GPUs. Source also labels memory 128 GiB; retain nominal 128 GB SKU and source units, without claiming measured physical byte capacity. 131072 MiB reported ROCm VRAM is unified exposed capacity, not dedicated VRAM or resident weight memory. CPU thread count, native resident memory, measured CPU/GPU core counts and concurrency are not established by these logs. PP and TG are separate fixed-work synthetic tests, not one chat request. PP512/PP2048 token counts are actual llama_decode input work; TG128 makes 128 single-token decode calls, without EOS-driven text sampling. Depth 0/4196/8392 is context populated before the timer, not new prompt tokens or capacity. Native timers use std::chrono high_resolution_clock nanoseconds. Load/context setup, warmup, per-repetition memory clearing and depth setup or restored depth cache precede timing. Timed functions include host token construction and llama_decode; PP synchronizes after its batches and TG synchronizes each token. Per-repetition rate is 1e9*(n_prompt+n_gen)/elapsed_ns; reported values are mean and sample SD of rates over 3 stated repetitions. The source initializes demand as n_prompt+n_gen+n_depth, but may reuse a larger context; actual allocation request and rounded capacity are unknown. No tokenization, HTTP/client timing or measured TTFT is provided; TTFT remains blank. No prompt_tokens/rate proxy is used. Native batch/KV settings are not in compact logs. Checked engine source defaults are batch 2048, ubatch 512, F16 KV, but defaults are not recorded run fields and remain reference-only. Requested offload 99/999 is not a verified number of model layers. HF links identify current reference metadata. Historical tested revision and hash remain unknown. Rounded native model tensor-storage size is neither disk file bytes nor measured resident memory. No weights were downloaded; no source software was executed. Raw rows (tokens/s mean±SD; d denotes preinitialized depth): vulkan-vs-rocm-sweep/bench-qwen36-a3b-bf16-rocm-shape1.log FA 0 pp512 479.55±1.69; FA 0 tg128 21.48±0.01; FA 1 pp512 484.01±4.4; FA 1 tg128 21.51±0.0 vulkan-vs-rocm-sweep/bench-qwen36-a3b-bf16-rocm-shape2.log FA 1 pp2048 474.69±2.76; FA 1 tg128 23.71±0.01; FA 1 pp2048 @ d4196 458.15±2.35; FA 1 tg128 @ d4196 23.39±0.01; FA 1 pp2048 @ d8392 440.02±0.67; FA 1 tg128 @ d8392 23.09±0.0 vulkan-vs-rocm-sweep/bench-qwen36-a3b-bf16-vulkan-shape1.log FA 0 pp512 306.85±1.0; FA 0 tg128 10.74±0.0; FA 1 pp512 305.21±0.87; FA 1 tg128 10.74±0.0 vulkan-vs-rocm-sweep/bench-qwen36-a3b-bf16-vulkan-shape2.log FA 1 pp2048 307.44±1.28; FA 1 tg128 10.73±0.0; FA 1 pp2048 @ d4196 291.88±1.28; FA 1 tg128 @ d4196 10.68±0.01; FA 1 pp2048 @ d8392 289.64±0.84; FA 1 tg128 @ d8392 10.64±0.0 Sources: https://huggingface.co/datasets/NorthstarAurora/strix-halo-bench-data/blob/4c24348922a9f2ff7e481956c0eb08fa28284227/vulkan-vs-rocm-sweep/bench-qwen36-a3b-bf16-rocm-shape1.log https://huggingface.co/datasets/NorthstarAurora/strix-halo-bench-data/blob/4c24348922a9f2ff7e481956c0eb08fa28284227/vulkan-vs-rocm-sweep/bench-qwen36-a3b-bf16-rocm-shape2.log https://huggingface.co/datasets/NorthstarAurora/strix-halo-bench-data/blob/4c24348922a9f2ff7e481956c0eb08fa28284227/vulkan-vs-rocm-sweep/bench-qwen36-a3b-bf16-vulkan-shape1.log https://huggingface.co/datasets/NorthstarAurora/strix-halo-bench-data/blob/4c24348922a9f2ff7e481956c0eb08fa28284227/vulkan-vs-rocm-sweep/bench-qwen36-a3b-bf16-vulkan-shape2.log https://github.com/h34v3nzc0dex/strix-halo-llm-finetune-guide/blob/46d062142eae35e36f0c1d0d91b393b59724432a/vulkan-vs-rocm-sweep/README.md https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF/blob/a483e9e6cbd595906af30beda3187c2663a1118c/BF16/Qwen3.6-35B-A3B-BF16-00001-of-00002.gguf Source QA: Compact logs start with device enumeration, not the command, despite the dataset card saying the first non-comment line is the command. Command shapes and repetitions come from the linked campaign guide and bench script. Raw build hashes are distinguished from guide release labels. No Zen 5c claim is copied from the dataset card. Timer source matching reported engine hash: https://github.com/ggml-org/llama.cpp/blob/a497476/tools/llama-bench/llama-bench.cpp Hardware source: https://huggingface.co/datasets/NorthstarAurora/strix-halo-bench-data/blob/4c24348922a9f2ff7e481956c0eb08fa28284227/README.md BF16 companion shard reference: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF/blob/a483e9e6cbd595906af30beda3187c2663a1118c/BF16/Qwen3.6-35B-A3B-BF16-00002-of-00002.gguf LlamaPerf structured rates are rounded to 0.1 tokens/s to match form precision; exact native means and SDs are retained above.

generation:

60.4 tokens/s

prompt processing (prefill):

942.2 tokens/s

quant:

UD-Q4_K_M (GGUF)

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Qwen3.6-35B-A3B UD-Q4_K_M native PP/TG Headline is Vulkan FA=1 shape 1, PP512 942.18±4.92 and TG128 60.39±0.22. Q4 Vulkan shape 2 includes both FA 0 and FA 1 although the README method abbreviates it to FA 1. Preserve all raw rows. Crucial confound: Q4 ROCm comparison logs are build 1acee6b (187), supported by sweep method 1acee6bf8, while Vulkan is a497476 (1)/b9296. Do not repeat the overview's identical-source backend comparison claim for these Q4 arms. ROCWMMA ON/OFF is a compile flag; runtime FA 0/1 is a separate axis. ROCm ON/OFF both use 1acee6bf8. Timer source at both reported revisions was fetched and is byte-identical. Source script binds tested filename Qwen3.6-35B-A3B-UD-Q4_K_M.gguf; the guide identifies unsloth downloads. Current pinned HF reference has that exact filename; it does not establish tested historical bytes. Raw size 20.60 GiB and params 34.66 B are retained. No multimodal or MTP speculation test is involved. Historical source-reported native llama-bench measurements by Paul Durkin / NorthstarAurora / h34v3nzc0dex. Evidence dataset revision 4c24348922a9f2ff7e481956c0eb08fa28284227; guide snapshot 46d062142eae35e36f0c1d0d91b393b59724432a. These are evidence snapshots, not engine or tested-model revisions. Exact measurement date is unpublished; the dataset existed by 2026-05-25. No new benchmark was run. Corsair AI Workstation 300, Ryzen AI MAX+ 395 / Radeon 8060S gfx1151, nominal 128 GB unified LPDDR5X-8000. Every retained raw log explicitly enumerates found 1 ROCm/Vulkan device and device 0, then shows that backend and positive GPU-layer offload. Thus participating physical GPU quantity = 1 per arm. Alternate ROCm/Vulkan builds run on the same APU, not two GPUs. Source also labels memory 128 GiB; retain nominal 128 GB SKU and source units, without claiming measured physical byte capacity. 131072 MiB reported ROCm VRAM is unified exposed capacity, not dedicated VRAM or resident weight memory. CPU thread count, native resident memory, measured CPU/GPU core counts and concurrency are not established by these logs. PP and TG are separate fixed-work synthetic tests, not one chat request. PP512/PP2048 token counts are actual llama_decode input work; TG128 makes 128 single-token decode calls, without EOS-driven text sampling. Depth 0/4196/8392 is context populated before the timer, not new prompt tokens or capacity. Native timers use std::chrono high_resolution_clock nanoseconds. Load/context setup, warmup, per-repetition memory clearing and depth setup or restored depth cache precede timing. Timed functions include host token construction and llama_decode; PP synchronizes after its batches and TG synchronizes each token. Per-repetition rate is 1e9*(n_prompt+n_gen)/elapsed_ns; reported values are mean and sample SD of rates over 3 stated repetitions. The source initializes demand as n_prompt+n_gen+n_depth, but may reuse a larger context; actual allocation request and rounded capacity are unknown. No tokenization, HTTP/client timing or measured TTFT is provided; TTFT remains blank. No prompt_tokens/rate proxy is used. Native batch/KV settings are not in compact logs. Checked engine source defaults are batch 2048, ubatch 512, F16 KV, but defaults are not recorded run fields and remain reference-only. Requested offload 99/999 is not a verified number of model layers. HF links identify current reference metadata. Historical tested revision and hash remain unknown. Rounded native model tensor-storage size is neither disk file bytes nor measured resident memory. No weights were downloaded; no source software was executed. Raw rows (tokens/s mean±SD; d denotes preinitialized depth): vulkan-vs-rocm-sweep/bench-qwen36-a3b-q4-vulkan-shape1.log FA 0 pp512 953.79±17.18; FA 0 tg128 60.69±0.15; FA 1 pp512 942.18±4.92; FA 1 tg128 60.39±0.22 vulkan-vs-rocm-sweep/bench-qwen36-a3b-q4-vulkan-shape2.log FA 0 pp2048 914.04±4.77; FA 0 tg128 60.95±0.07; FA 0 pp2048 @ d4196 848.01±1.25; FA 0 tg128 @ d4196 57.71±0.1; FA 0 pp2048 @ d8392 784.85±2.02; FA 0 tg128 @ d8392 54.95±0.1; FA 1 pp2048 921.04±2.97; FA 1 tg128 60.64±0.02; FA 1 pp2048 @ d4196 876.8±4.53; FA 1 tg128 @ d4196 58.24±0.09; FA 1 pp2048 @ d8392 835.33±2.28; FA 1 tg128 @ d8392 57.13±0.09 rocwmma-fattn-sweep/bench-qwen36-35b-a3b-q4-fattn-off.log FA 0 pp512 1023.75±8.66; FA 0 tg128 49.63±0.33; FA 1 pp512 1014.32±9.65; FA 1 tg128 49.58±0.09; FA 0 pp2048 997.11±8.12; FA 0 tg128 49.54±0.1; FA 0 pp2048 @ d4196 853.85±1.04; FA 0 tg128 @ d4196 43.93±0.21; FA 0 pp2048 @ d8392 753.86±0.68; FA 0 tg128 @ d8392 39.26±0.03; FA 1 pp2048 983.86±2.91; FA 1 tg128 49.7±0.09; FA 1 pp2048 @ d4196 881.86±0.64; FA 1 tg128 @ d4196 48.64±0.24; FA 1 pp2048 @ d8392 815.7±0.57; FA 1 tg128 @ d8392 46.73±0.02 rocwmma-fattn-sweep/bench-qwen36-35b-a3b-q4-fattn-on.log FA 0 pp512 1028.24±22.95; FA 0 tg128 49.45±0.1; FA 1 pp512 934.0±7.88; FA 1 tg128 49.52±0.08; FA 0 pp2048 988.33±7.57; FA 0 tg128 49.41±0.09; FA 0 pp2048 @ d4196 847.22±45.1; FA 0 tg128 @ d4196 43.81±0.18; FA 0 pp2048 @ d8392 750.44±1.65; FA 0 tg128 @ d8392 39.14±0.01; FA 1 pp2048 813.71±2.37; FA 1 tg128 49.43±0.08; FA 1 pp2048 @ d4196 467.28±0.38; FA 1 tg128 @ d4196 46.85±0.22; FA 1 pp2048 @ d8392 332.32±0.72; FA 1 tg128 @ d8392 44.08±0.01 Sources: https://huggingface.co/datasets/NorthstarAurora/strix-halo-bench-data/blob/4c24348922a9f2ff7e481956c0eb08fa28284227/vulkan-vs-rocm-sweep/bench-qwen36-a3b-q4-vulkan-shape1.log https://huggingface.co/datasets/NorthstarAurora/strix-halo-bench-data/blob/4c24348922a9f2ff7e481956c0eb08fa28284227/vulkan-vs-rocm-sweep/bench-qwen36-a3b-q4-vulkan-shape2.log https://huggingface.co/datasets/NorthstarAurora/strix-halo-bench-data/blob/4c24348922a9f2ff7e481956c0eb08fa28284227/rocwmma-fattn-sweep/bench-qwen36-35b-a3b-q4-fattn-off.log https://huggingface.co/datasets/NorthstarAurora/strix-halo-bench-data/blob/4c24348922a9f2ff7e481956c0eb08fa28284227/rocwmma-fattn-sweep/bench-qwen36-35b-a3b-q4-fattn-on.log https://github.com/h34v3nzc0dex/strix-halo-llm-finetune-guide/blob/46d062142eae35e36f0c1d0d91b393b59724432a/vulkan-vs-rocm-sweep/README.md https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF/blob/a483e9e6cbd595906af30beda3187c2663a1118c/Qwen3.6-35B-A3B-UD-Q4_K_M.gguf Source QA: Compact logs start with device enumeration, not the command, despite the dataset card saying the first non-comment line is the command. Command shapes and repetitions come from the linked campaign guide and bench script. Raw build hashes are distinguished from guide release labels. No Zen 5c claim is copied from the dataset card. Timer source matching reported engine hash: https://github.com/ggml-org/llama.cpp/blob/a497476/tools/llama-bench/llama-bench.cpp; https://github.com/ggml-org/llama.cpp/blob/1acee6bf8/tools/llama-bench/llama-bench.cpp Hardware source: https://huggingface.co/datasets/NorthstarAurora/strix-halo-bench-data/blob/4c24348922a9f2ff7e481956c0eb08fa28284227/README.md LlamaPerf structured rates are rounded to 0.1 tokens/s to match form precision; exact native means and SDs are retained above.