llamaperf
Oct 13, 2026
Throughput
60.4 t/s gen · 942.2 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
UD-Q4_K_M (GGUF)
Flash Attention
on
MTP (Multi-Token Prediction)
off
System RAM
128 GB

Summary

Qwen3.6-35B-A3B UD-Q4_K_M native PP/TG Headline is Vulkan FA=1 shape 1, PP512 942.18±4.92 and TG128 60.39±0.22. Q4 Vulkan shape 2 includes both FA 0 and FA 1 although the README method abbreviates it to FA 1. Preserve all raw rows. Crucial confound: Q4 ROCm comparison logs are build 1acee6b (187), supported by sweep method 1acee6bf8, while Vulkan is a497476 (1)/b9296. Do not repeat the overview's identical-source backend comparison claim for these Q4 arms. ROCWMMA ON/OFF is a compile flag; runtime FA 0/1 is a separate axis. ROCm ON/OFF both use 1acee6bf8. Timer source at both reported revisions was fetched and is byte-identical. Source script binds tested filename Qwen3.6-35B-A3B-UD-Q4_K_M.gguf; the guide identifies unsloth downloads. Current pinned HF reference has that exact filename; it does not establish tested historical bytes. Raw size 20.60 GiB and params 34.66 B are retained. No multimodal or MTP speculation test is involved. Historical source-reported native llama-bench measurements by Paul Durkin / NorthstarAurora / h34v3nzc0dex. Evidence dataset revision 4c24348922a9f2ff7e481956c0eb08fa28284227; guide snapshot 46d062142eae35e36f0c1d0d91b393b59724432a. These are evidence snapshots, not engine or tested-model revisions. Exact measurement date is unpublished; the dataset existed by 2026-05-25. No new benchmark was run. Corsair AI Workstation 300, Ryzen AI MAX+ 395 / Radeon 8060S gfx1151, nominal 128 GB unified LPDDR5X-8000. Every retained raw log explicitly enumerates found 1 ROCm/Vulkan device and device 0, then shows that backend and positive GPU-layer offload. Thus participating physical GPU quantity = 1 per arm. Alternate ROCm/Vulkan builds run on the same APU, not two GPUs. Source also labels memory 128 GiB; retain nominal 128 GB SKU and source units, without claiming measured physical byte capacity. 131072 MiB reported ROCm VRAM is unified exposed capacity, not dedicated VRAM or resident weight memory. CPU thread count, native resident memory, measured CPU/GPU core counts and concurrency are not established by these logs. PP and TG are separate fixed-work synthetic tests, not one chat request. PP512/PP2048 token counts are actual llama_decode input work; TG128 makes 128 single-token decode calls, without EOS-driven text sampling. Depth 0/4196/8392 is context populated before the timer, not new prompt tokens or capacity. Native timers use std::chrono high_resolution_clock nanoseconds. Load/context setup, warmup, per-repetition memory clearing and depth setup or restored depth cache precede timing. Timed functions include host token construction and llama_decode; PP synchronizes after its batches and TG synchronizes each token. Per-repetition rate is 1e9*(n_prompt+n_gen)/elapsed_ns; reported values are mean and sample SD of rates over 3 stated repetitions. The source initializes demand as n_prompt+n_gen+n_depth, but may reuse a larger context; actual allocation request and rounded capacity are unknown. No tokenization, HTTP/client timing or measured TTFT is provided; TTFT remains blank. No prompt_tokens/rate proxy is used. Native batch/KV settings are not in compact logs. Checked engine source defaults are batch 2048, ubatch 512, F16 KV, but defaults are not recorded run fields and remain reference-only. Requested offload 99/999 is not a verified number of model layers. HF links identify current reference metadata. Historical tested revision and hash remain unknown. Rounded native model tensor-storage size is neither disk file bytes nor measured resident memory. No weights were downloaded; no source software was executed. Raw rows (tokens/s mean±SD; d denotes preinitialized depth): vulkan-vs-rocm-sweep/bench-qwen36-a3b-q4-vulkan-shape1.log FA 0 pp512 953.79±17.18; FA 0 tg128 60.69±0.15; FA 1 pp512 942.18±4.92; FA 1 tg128 60.39±0.22 vulkan-vs-rocm-sweep/bench-qwen36-a3b-q4-vulkan-shape2.log FA 0 pp2048 914.04±4.77; FA 0 tg128 60.95±0.07; FA 0 pp2048 @ d4196 848.01±1.25; FA 0 tg128 @ d4196 57.71±0.1; FA 0 pp2048 @ d8392 784.85±2.02; FA 0 tg128 @ d8392 54.95±0.1; FA 1 pp2048 921.04±2.97; FA 1 tg128 60.64±0.02; FA 1 pp2048 @ d4196 876.8±4.53; FA 1 tg128 @ d4196 58.24±0.09; FA 1 pp2048 @ d8392 835.33±2.28; FA 1 tg128 @ d8392 57.13±0.09 rocwmma-fattn-sweep/bench-qwen36-35b-a3b-q4-fattn-off.log FA 0 pp512 1023.75±8.66; FA 0 tg128 49.63±0.33; FA 1 pp512 1014.32±9.65; FA 1 tg128 49.58±0.09; FA 0 pp2048 997.11±8.12; FA 0 tg128 49.54±0.1; FA 0 pp2048 @ d4196 853.85±1.04; FA 0 tg128 @ d4196 43.93±0.21; FA 0 pp2048 @ d8392 753.86±0.68; FA 0 tg128 @ d8392 39.26±0.03; FA 1 pp2048 983.86±2.91; FA 1 tg128 49.7±0.09; FA 1 pp2048 @ d4196 881.86±0.64; FA 1 tg128 @ d4196 48.64±0.24; FA 1 pp2048 @ d8392 815.7±0.57; FA 1 tg128 @ d8392 46.73±0.02 rocwmma-fattn-sweep/bench-qwen36-35b-a3b-q4-fattn-on.log FA 0 pp512 1028.24±22.95; FA 0 tg128 49.45±0.1; FA 1 pp512 934.0±7.88; FA 1 tg128 49.52±0.08; FA 0 pp2048 988.33±7.57; FA 0 tg128 49.41±0.09; FA 0 pp2048 @ d4196 847.22±45.1; FA 0 tg128 @ d4196 43.81±0.18; FA 0 pp2048 @ d8392 750.44±1.65; FA 0 tg128 @ d8392 39.14±0.01; FA 1 pp2048 813.71±2.37; FA 1 tg128 49.43±0.08; FA 1 pp2048 @ d4196 467.28±0.38; FA 1 tg128 @ d4196 46.85±0.22; FA 1 pp2048 @ d8392 332.32±0.72; FA 1 tg128 @ d8392 44.08±0.01 Sources: https://huggingface.co/datasets/NorthstarAurora/strix-halo-bench-data/blob/4c24348922a9f2ff7e481956c0eb08fa28284227/vulkan-vs-rocm-sweep/bench-qwen36-a3b-q4-vulkan-shape1.log https://huggingface.co/datasets/NorthstarAurora/strix-halo-bench-data/blob/4c24348922a9f2ff7e481956c0eb08fa28284227/vulkan-vs-rocm-sweep/bench-qwen36-a3b-q4-vulkan-shape2.log https://huggingface.co/datasets/NorthstarAurora/strix-halo-bench-data/blob/4c24348922a9f2ff7e481956c0eb08fa28284227/rocwmma-fattn-sweep/bench-qwen36-35b-a3b-q4-fattn-off.log https://huggingface.co/datasets/NorthstarAurora/strix-halo-bench-data/blob/4c24348922a9f2ff7e481956c0eb08fa28284227/rocwmma-fattn-sweep/bench-qwen36-35b-a3b-q4-fattn-on.log https://github.com/h34v3nzc0dex/strix-halo-llm-finetune-guide/blob/46d062142eae35e36f0c1d0d91b393b59724432a/vulkan-vs-rocm-sweep/README.md https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF/blob/a483e9e6cbd595906af30beda3187c2663a1118c/Qwen3.6-35B-A3B-UD-Q4_K_M.gguf Source QA: Compact logs start with device enumeration, not the command, despite the dataset card saying the first non-comment line is the command. Command shapes and repetitions come from the linked campaign guide and bench script. Raw build hashes are distinguished from guide release labels. No Zen 5c claim is copied from the dataset card. Timer source matching reported engine hash: https://github.com/ggml-org/llama.cpp/blob/a497476/tools/llama-bench/llama-bench.cpp; https://github.com/ggml-org/llama.cpp/blob/1acee6bf8/tools/llama-bench/llama-bench.cpp Hardware source: https://huggingface.co/datasets/NorthstarAurora/strix-halo-bench-data/blob/4c24348922a9f2ff7e481956c0eb08fa28284227/README.md LlamaPerf structured rates are rounded to 0.1 tokens/s to match form precision; exact native means and SDs are retained above.