Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3-Coder-30B-A3B-Instruct at 70.25 t/s generation on an AMD Strix Halo (GMKTEC EVO-X2) with 128GB unified memory.
Setup is llama.cpp (ROCm 7 RC, rocWMMA) with Q4_K_M GGUF, 8K context, all 99 layers offloaded to GPU, 2 parallel slots.
Short prompt (9 tokens) prefill was 4.94 t/s; medium (51 tokens) 28.29 t/s and 71.00 t/s generation; large (88 tokens) 46.81 t/s and 68.09 t/s generation. User rates the setup 5/5 and notes generation is 62% faster than a previous Llama-2-7B baseline.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
long-contextcodingagentic
User reports Qwen3.6-35B-A3B at around 25 tok/s on a Strix Halo 128GB box (Minisforum MS-S1 MAX, Ryzen AI Max+ 395).
Setup is llama.cpp b8890 with UD-Q8_K_XL GGUF and q8_0 KV cache, Vulkan RADV backend, 524288 context, 3 parallel slots, running under 100W.
Live llama.cpp metrics at capture: 153562 tokens max context, 322.685 prompt tokens/s, 24.0673 predicted tokens/s. User notes long context needs architecture, quant, KV cache and backend to line up, and plans to test UD-Q3_K_XL next.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User benchmarks Qwen3.5 122B A10B at 19.17 t/s generation on a Strix Halo 128GB.
Setup is llama.cpp with UD-Q6_K_XL GGUF, ROCm 7.2.3, full GPU offload, no MTP speculation. The model is 106.04 GiB.
With MTP enabled the same model reaches 26.86 t/s. Prompt processing peaks at 266.48 t/s with batch size 512. A dense Gemma 4 31B Q4_K_M comparison runs at 10.48 t/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
long-context
User reports a third-party sourced prefill measurement of Qwen3.8-Flash-Next at 1057.53 t/s on a single Strix Halo 128GB system.
Setup is Gufo with Unsloth UD-Q4_K_XL GGUF weights, a 256000-token prompt with zero cached prefix tokens, 512-token prefill chunks, and MTP enabled with a shared Q8 MTP head and draft ceiling 7.
The row is prefill-only and generates one setup token, so no generation throughput is reported. One observation per cell on a synthetic counting workload; no repeats, uncertainty interval, TTFT or row-level memory peak are available.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next at 34.19 t/s decode and 1541.16 t/s prefill on two AMD Strix Halo 128GB hosts, one Radeon 8060S gfx1151 GPU per host, tensor parallel world size 2 over RDMA/InfiniBand.
Setup is the Gufo RDMA fork with Unsloth UD-Q4_K_XL GGUF (four shards), configured context capacity 262144 tokens, greedy temperature 0, thinking off, speculative decoding and MTP off; the measured input was 2040 tokens, not 262144.
Single warmed prose sample, one session, HTTP OpenAI chat-completions SSE; source-maintainer measurement, not independently replicated, with no standard deviation. KV-cache precision, Flash Attention setting, measured resident RAM and verified binary identity are unknown; the prose labels revision ea76571 while the raw artifact records 795407e30460aeb730aaff2316739baa3a1348e2 with dirty=true.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
agenticlong-context
User reports DeepSeek-V4-Flash-0731 (284B MoE, 13B active) at 28.91 t/s generation and 261.7 t/s prompt processing on a single AMD Ryzen AI MAX+ 395 with 128GB unified memory.
Setup is a llama.cpp fork (Nathanw1014/llama.cpp) with Vulkan/RADV backend, UD-IQ2_XXS quant (2.06 bpw), DSpark draft model at n=3, 16384 context, 120 GiB GTT limit.
Best result is IQ2_XXS + DSpark n=3 at 28.91 t/s, reaching 90% of Lucebox's 32.0 t/s with 6 experts instead of 4. The fork beats upstream by 49% on prompt processing at 8192 context. IQ2 with draft model sits at the memory limit and is a measurement setup, not a production state.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash at 484.62 t/s prefill and 16.28 t/s decode on a Strix Halo 128GB (Ryzen AI MAX+ 395, Radeon 8060S, gfx1151).
Setup is Gufo with an IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8 mixed GGUF, one session, speculative decoding disabled, thinking off, temperature 0, seed 1, nominal pp2048/tg128 (2,035 prompt tokens, 128 generated).
One warmed sample; no standard deviation reported. The maintainer notes target-model quality parity is unestablished and build-sensitive differences are under investigation. Power, KV-cache type, Flash Attention and peak memory were not reported.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8 Flash-Next decode at 47.6 t/s on Strix Halo 128GB, with prefill of 994 t/s at 2K context.
Setup is a custom Windows attempt with speculative decoding; prefill reaches 1,026 t/s at 8K and 1,015 t/s at 32K, and decode without speculation is 34.0 t/s short, 30.7 t/s at 32K and 28.6 t/s at 128K.
User compares against Linux and Windows results from Halogen, gufo, strix-alloy and strixllama, and reports 30/30 on a 68K truncation test. No repo shared yet.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next at 42.0 t/s generation and 887 t/s prefill on a GMKtec EVO-X2 with AMD Ryzen AI Max+ 395 (Strix Halo, 128GB unified, 64GB GPU carve-out).
Setup is Strata 0.1.40.2 (HIP/ROCm) with Unsloth UD-IQ4_XS GGUF, int8 KV cache, 32,768 resident KV tokens, --spec 4 with MTP enabled, and a fresh 17,844-token prompt generating 300 tokens.
Single-machine community self-report, not independently reproduced; MTP/speculative decoding is on, so this is not a non-speculative baseline. Chip power was about 121 W (peaks 138 W) and GPU clock median 2.75 GHz.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next at 21.05 tokens/s engine-reported output decode on a Strix Halo 128GB (Ryzen AI MAX+ 395 / Radeon 8060S).
Setup is Gufo with UD-Q4_K_XL GGUF weights, configured context capacity 131584 tokens, one slot, MTP off, thinking off, temperature 0.
The run read an actual input of 127525 tokens and produced 420 output tokens; reported prefill duration was 93.24 seconds and whole-request wall time 113.47 seconds, with no derived prefill rate submitted because raw counter semantics were not independently verified. One cold sample; cached repeats excluded.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codinglong-context
User reports Qwen3.8-Flash-Next at 38.3 t/s generation and 857 t/s prefill on a Strix Halo 128GB.
Setup is a ROCm llama.cpp branch with the BALANCED-2.1 GGUF quant (78.3 GB) and MTP speculative decoding, single card, 8K context.
The 38.3 t/s is the short-answer figure at 8K; code writing reached 46.7 t/s and long prose 30.1 t/s at the same depth. Prefill fell from 857 t/s at 8K to 301 t/s at 256K, and generation from 38.3 to 25.2 t/s. GPU memory stayed at 68-70 GiB across the full window.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codinglong-context
User reports Qwen3.8-27B at 42.4 t/s writing code on one Strix Halo 128GB, with 404 t/s prompt reading at 8K context.
Setup is the paoai-strix-engine strix-v4 Vulkan build with a 15 GB ROCmFP4 GGUF, MTP draft head for the Chat profile, DFlash2 adaptive drafting for Code and Long, and an f16 KV cache for Long.
Three serving profiles were measured at 8K, 32K, 64K, 128K and 256K with fresh prompts and prompt cache off; the 256K window was verified on all three, and the Code and Long profiles scored 100 on a coding exam. A 55-task quality bench scored 83.2 for Chat.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8 Flash-Next at 60+ tok/s decode and 2000+ tok/s prefill on Strix Halo 128GB plus an R9700 32GB.
Setup is a modified llama.cpp (llama-halo-hybrid) with Q4_K_XL weights, splitting dense layers and KV cache to the GPU while the APU runs the rest, supporting full 256k context.
The user says it beats DGX Spark running the same model, with the Swift-1.5 variant performing slightly better, and recommends Strix 128GB plus R9700 over Gorgon Halo for price and performance.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
agentictool-use
User reports gpt-oss-120b at 54.11 t/s generation and 645.07 t/s prompt processing on a GMKtec EVO-X2 with AMD Ryzen AI Max+ 395 (Strix Halo) and 128GB unified memory.
Setup is llama.cpp build 11456 with MXFP4 GGUF weights, Vulkan (Mesa RADV) backend, flash attention on, whole model on the iGPU, no speculative decoding, one request at a time.
Three runs after a warm-up gave 54.11, 54.64 and 54.50 t/s generation with 645.07, 656.81 and 657.14 t/s prompt processing on an 839-token prompt. Two small embedding and reranker models were loaded but idle. The user runs this model for agentic tool use and research.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingagentic
User reports Qwen 3.5 35B at 49.7 tok/s on a Strix Halo 128GB machine.
Setup is llama.cpp via llama-swap with a HuggingFace GGUF Q8 quant.
The same model completed the coding benchmark under Ollama but produced off-spec output; under llama-swap it completed with a full Rails app. Other local models on the same machine: Qwen 3 Coder Next 72.9 tok/s, GPT OSS 20B 78.3 tok/s, Gemma 4 27B 47.6 tok/s, GLM 4.7 Flash 47.4 tok/s, Qwen 3.5 122B 23.1 tok/s, Llama 4 Scout 17.5 tok/s, Qwen 3 32B 11.7 tok/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks Qwen3.8-Flash-Next at 28.4 t/s decode and 314 t/s prefill on a Minisforum S1 Max with AMD Ryzen AI Max+ 395 and 128 GB unified memory.
Setup is llama.cpp (Vulkan, Unsloth build) with UD-Q4_K_XL weights and MTP speculative decoding at 56% accept rate.
The repo also lists community engine comparisons on the same hardware: gufo 34.1 t/s decode and 1,075 t/s prefill, Halogen 39.4/1,191, strixllama 29.7/716, and llama.cpp ROCm 20.5/285. The GGUF checkpoint uses a shared BF16 PLE of about 95 GiB that should stay NVMe-backed via mmap on 128 GB machines.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next at 1700.52 t/s prompt processing and 25.95 t/s generation on an AMD Strix Halo 128GB.
Setup is Gufo with an Unsloth UD-Q4_K_XL GGUF, single-user autoregressive with no speculative decoding, greedy sampling, thinking off, at cached depth zero (prompt_n 2040, predicted_n 128).
This is an engine-maintainer measurement, not an independent reproduction, and the measured binary was built from a dirty source tree. Allocated context capacity is 133760 tokens, not prompt length. MTP, aggregate concurrency and cached-depth rows are separate workloads.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8 27B at 10 t/s decode on a Strix Halo 128GB using gufo, the same speed they saw with llama.cpp.
Setup is gufo with default parameters except the thinking spec, on a software design task that required reading an existing repository.
The user says the model did great work but took 90 minutes where a frontier model took 5, and asks how to speed the task up.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagenticlong-context
User reports Qwen3.8 Flash-Next 125B MoE decoding at 64 tok/s and prefilling ~1,500 tok/s on a Strix Halo tablet, with the NPU handling search, dup scan, decisions and screening.
Setup is the Halogen engine with 262k context, fully local; the GPU still does the thinking and the NPU changed what tokens got spent on, at ~7% iGPU cost when they overlap.
First token lands in ~0.03s, about 43x faster than a cloud call measured side by side and still 7x with a second agent hammering the server; a 194k session compacted in ~50s versus 166 on the main model with 97% cache hit.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagenticlong-context
User (Yamz Labs) reports Qwen3.8-Flash-Next at 32.7 tok/s decode without speculative decoding and 1,412 tok/s prefill at 4K on a single Strix Halo mini PC with their Kyojin engine.
Setup is a single AMD Ryzen AI Max+ 395 with 128GB unified memory, 95GB EXL3 weights, Kyojin on ExLlamaV3, plain decoding with no speculation.
With speculative decoding decode reaches 44 to 59 tok/s depending on the task (chat ~47, code ~58, copy-heavy edits ~60). Prefill stays near flat at 1,486 tok/s at 32K and 1,367 at 128K. The authors say Halogen 0.16.2 is faster at 39.8 tok/s plain.