Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8 Flash-Next at 20 t/s generation on a Strix Halo 128GB laptop with 64GB RAM.
Setup is llama.cpp via lemonade with Vulkan on Windows, GSQ-RCO GGUF quant, 16k context, Q4 KV cache, and lazy load enabled. The model uses about 30GB of GPU memory but also consumes roughly 20GB of CPU memory, which the user suspects is the n-gram table not staying on SSD.
The user notes the model loaded and worked rather well, but a default prompt ran out of the 16k context due to extended thinking.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
agentictool-uselong-context
User reports Qwen3.8-Flash-Next 125B at 43.0 tok/s decode and 656 tok/s prefill on a single Ryzen AI Max+ 395 (Strix Halo, 128 GB unified memory).
Setup is a llama.cpp Vulkan fork with 57 patches, Unsloth UD-Q4_K_XL (111 GB), 8k context, MTP speculative decoding, single request, no CPU offload.
At 32k context decode is 40.1 tok/s and prefill 585 tok/s. Inside a tool call decode is about 55 tok/s (was 25), first token after a short tool result 0.4 s (was 1.5 s), and the agent loop over 12 shell commands runs at 16.2 tokens per wall-clock second (was 11.1). KL divergence against full-precision reference logits is 0.116 with 92.2% same top-1 token. The user notes the runtime is reproducible bit for bit and that a PaoAI BALANCED-2.1 file ran 6-9% faster but with about 5x the KL divergence (0.589).
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingagentic
User reports Qwen3.8-Flash-Next (125B-A6B, Unsloth UD-IQ4_XS) at 46 tok/s short and 42 tok/s at 86K context with MTP on a single conversation, on a Ryzen AI Max+ 395 (Strix Halo 128GB) under Windows.
Setup is Rulith Inference 0.4 (formerly Strix Llama) with MTP speculative decoding and 2-token drafts. Prefill is 1,237 tok/s over a 95.6K-token prompt. With MTP off, 8 conversations of ~40K tokens each reach 91 tok/s combined; short prompts with typical sampling reach ~100 tok/s.
The user runs it as a local worker for coding agents, seeing 30+ tok/s with one agent, 50+ combined with two, and 60+ with three. A caveat is that the local model guesses more (IDs, file paths, field names), so output goes through review.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8-27B at around 30 t/s on a Bosgame M5 with Ryzen AI Max+ 395 and 128 GB, used for coding.
Setup is a Q4 quant; the user also runs Gemma4 26B A4B at around 90 t/s for other tasks.
The user asks whether switching to Qwen3.8-Flash-Next is worth it, noting a 4-bit quant of it is around 90 GB and would require dropping the second model.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
long-contextcodingagentic
User benchmarks Qwen3.8 Flash-Next on a ROG Flow Z13 2025 with Strix Halo 128GB, comparing Halogen, Gufo and Rulith at 60W and 93W.
Gufo 0.5.0 with UD-Q4_K_XL and MTP Q8_0 draft 7 reaches 1343 tok/s prefill and 33.6 tok/s decode at 200K context at 93W, and 1156 tok/s prefill with 33.0 tok/s decode at 60W.
Halogen 0.16.0 is fastest for prefill, hitting 1694 tok/s at 200K and 41.16 tok/s decode at 93W, while Rulith on Windows is fastest at short context with 54.7 tok/s decode at 1K but falls to 32.55 tok/s at 200K. The user notes the comparison is not apples-to-apples because model formats and MTP setups differ between backends.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codinglong-context
User benchmarks Qwen3.8-Flash-Next on an AMD Strix Halo 128GB (ASUS ROG Flow Z13 GZ302) at 70 W TDP, comparing several engines. The gufo engine with UD-Q4_K_XL weights reaches 32.9 t/s decode and 1,033 t/s prefill at 64k context (32k prompt), with 74% MTP acceptance and 14/14 retrieval.
Setup uses gufo (ROCm 7.2.4) with UD-Q4_K_XL GGUF weights and MTP speculative decoding, run via LlamaStash on Arch Linux.
Halogen 0.14.0 with native .hgn weights is fastest at 39.3 t/s decode and 1,045 t/s prefill, but is closed source and Docker-only. gufo is open source and loads 4x faster from cold. At 128k context (64k prompt), gufo gets 32.3 t/s decode and 1,047 t/s prefill; at 256k context (130k prompt), 27.9 t/s decode and 997 t/s prefill. In 10 Aider polyglot Python exercises, gufo passes 10/10 in 36.0 min, versus Halogen 21.5 min and CIRU 24.6 min.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B at 70.22 tok/s on a Strix Halo device, reproducing Gufo's published 70.56 tok/s figure.
Setup is Gufo 0.4.0 with UD-Q4_K_XL GGUF weights and a DFlash2 Q4_K_M draft model, greedy decoding, thinking off, 128 output tokens, prompt cache off. The 70 tok/s figure comes from a repetitive prompt ("Write the word red exactly 1000 times") where speculative decoding accepts nearly every draft token.
On nine ordinary prompts the median is 39.4 tok/s, ranging from 22 to 52. With 8 users the aggregated figure is 122.6 tok/s but wall-clock token delivery is 82 tok/s on the repetitive prompt and 52 on normal prompts. Without the draft model Gufo's docs put it around 12 tok/s. In a head-to-head against halogen 0.13.8 on Qwen3.8 Flash-Next, halogen averaged 43.9 tok/s vs Gufo's 38.2 on normal prompts, and 76.6 vs 63.1 tok/s with 4 users; Gufo was faster at cold prefill (1495 vs 1288 tok/s) and on the repetitive prompt (87.4 vs 56.9 tok/s).
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B at 39.4 t/s median on normal prompts on a Strix Halo device.
Setup is Gufo 0.4.0 with UD-Q4_K_XL GGUF and a DFlash2 Q4_K_M draft model, single user, speculative decoding on.
The 70.22 t/s figure comes from a repetitive prompt and is a ceiling; normal prompts range from 22 to 52 t/s. In a head-to-head with halogen 0.13.8 on identical hardware, halogen was about 13% faster for one user and 18% faster with four, while Gufo was 16% faster at prompt processing and much faster on the repetitive prompt.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User compares Qwen3.8 Flash-Next Q4_K_M at 160,000 context across a Strix Halo and a six-GPU machine. On Strix Halo with mainline llama.cpp 0.4.1 the run reaches 113.48 t/s prefill and 7.06 t/s generation.
Setup is llama.cpp with Q4_K_M weights, 103.68 GiB model, fully in the Strix Halo's 128GB unified memory.
At the same context the Gufo fork gives 1227.12 t/s prefill and 22.04 t/s generation, Halo Box gives 587.99 t/s and 21.24 t/s, and the pile of GPUs (2x RTX 5070 Ti, 2x RTX 5060 Ti 16GB, 2x RX 7900 XTX) gives 215.73 t/s and 16.29 t/s. User concludes Strix Halo shines for this model.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B at 26.47 t/s on a GMKtec NucBox EVO-X2 with AMD Ryzen AI Max+ 395 (Strix Halo) and 128 GB LPDDR5X-8000.
Setup is LM Studio's llama.cpp Vulkan build with Q4_K_M (lmstudio-community, 16.80 GB) and the model's built-in MTP speculative-decoding head on, thinking off.
Other quants measured with MTP: UD-Q4_K_XL (unsloth, 17.92 GB) 26.26 t/s, UD-Q8_K_XL (unsloth, 31.45 GB) 15.20 t/s, BF16 (unsloth, 54.66 GB) 10.50 t/s. Memory bandwidth measured at about 239 GB/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B at 7.24 tok/s on a Strix Halo 128GB (Ryzen AI Max+ 395, Radeon 8060S).
Setup is llama.cpp build 9867 with Vulkan backend, UD-Q8_K_XL GGUF, -ngl 999 -fa on, single request, no MTP.
With MTP self-speculative decoding the same configuration reaches 16.46 tok/s on code. Q6 baseline without MTP runs 8.43-8.44 tok/s and Q6 with MTP 18.67-18.72 tok/s. At four concurrent slots, Q8 aggregate is 21.75 tok/s without MTP and 16.94 with.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
agentictool-use
User reports Qwen3.8 Flash-Next running at approximately 40-50 tok/s on a Ryzen Strix Halo PC with 128GB unified RAM.
Setup is gufo with Q4 weights and 256k context, supporting 2 concurrent sessions.
The user notes the model impressed them with its context understanding and tooling, and that it provides capability comparable to good API models.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports MiMo 2.6 Flash-RL at 26.1 tok/s decode (tg128) and ~310 tok/s prefill (pp4096) on a Strix Halo 128GB machine.
Setup is llama.cpp (Vulkan, Radeon 8060S) with a custom IQ2_M-class GGUF at 2.76 bpw and q8_0 KV cache, 32768 context, -ub 2048. Prefill drops to 193 tok/s at the default -ub 512.
The 100.4 GiB quant is measured against the native MXFP4 GGUF: KLD 0.164 mean, 88.2% same top-1 token, PPL ratio 1.128. MTP self-speculation is included in the file but does not speed up decode (draft acceptance ~50%), so speculation is off.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 Flash-Next at 57 t/s decode and 1239 tok/s encode on Strix Halo 128GB.
Setup uses MTP speculative decoding; the run processed 6204 chunks in 119.0 s.
User says the prefill speed is twice that of the fastest Strix Halo-specific llama.cpp fork they have used, and greater still versus mainline llama.cpp.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
agenticlong-contextvisioncoding
User reports Qwen3.8-Flash-Next (125B MoE, ~6B active) at 44.9 tok/s decode after a short question and 1217 t/s prefill on a 95.6K-token prompt, on a single Ryzen AI Max+ 395 (Strix Halo, 128 GB, 96 GB GPU memory in BIOS) running Windows.
Setup is a patched llama.cpp runtime (Strix Llama 0.3.0) with Unsloth's UD-IQ4_XS GGUF quant (94 GB), 256K context, MTP speculative decoding on (~2.9 tokens per pass), and an OpenAI-compatible API.
Decode drops to 35.8 tok/s at 86K context; with MTP off it is ~26 tok/s after a short question and ~23 tok/s at 86K context. Three and four concurrent conversations reach 58.8 and 62.6 tok/s combined. Decode is memory-bandwidth-bound (~5.8 GB of weights read per token at ~215 GB/s).
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports running Qwen3.8 Flash-Next on a Minisforum Strix Halo 128GB box, sustaining 40 to 50 tokens per second on a one-shot meteor game smoke test.
Setup uses Vulkan rather than ROCm, with MTP speculative decoding at depth 4 and context set to 128K minimum.
The user notes the box cost £3500 and that the DGX Spark is about 20% more expensive with similar 256GB/s memory bandwidth.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingmathtool-useagentic
User reports Qwen 3.8 27B at 14.02 tok/s unassisted decode on an AMD Strix Halo (Ryzen AI Max+ 395, 128 GB).
Setup is a ROCmFPX-enabled llama.cpp fork with ROCmFP4_FAST quant (4.26 bpw, 13.55 GiB GGUF) and Asymmetric TurboQuant KV cache at 32K context.
With MTP speculative decoding, decode reaches 34.82 tok/s (2.84x) at greedy sampling; deep spec n7/p0.35 reaches 36.04 tok/s on JSON and code. At temperature 0.8 draft acceptance collapses to about 25% and speculative decoding returns close to the unassisted 14.02 tok/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
agenticcodingtool-use
User reports Qwen3.6 27B at 19.63 t/s on an AMD Strix Halo 128GB, within 2.3% of a DGX Spark's 20.1 t/s on the MLPerf v6.1 Edge Agentic benchmark.
Setup is the official MLPerf Edge Agentic harness with NVFP4 quant, same engine and quant recipe on both boxes, using 1,007 recorded agentic coding turns and BFCL function-calling accuracy gates.
A separate native Windows/Linux build for Qwen3.8 27B NVFP4 was verified on a Framework Desktop at 17.3 t/s on gfx1151 with an HA-20 BFCL score of 84.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
coding
User reports a Qwen model running on a Bosgame M5 with 128GB (Strix Halo) under ROCm, reading a VS Code Copilot prompt.
Setup is a franken-llama build assembled from various sources; prompt processing at 32k context reaches 950 t/s.
Generation speed is described as nothing to write home about, with no figure given.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B Q4 at up to 70.56 t/s generation and 656.33 t/s prompt processing on a Strix Halo 128GB.
Setup is the Gufo inference engine, a new vertically integrated engine built specifically for Strix Halo hardware.
At concurrency 8 the model reaches 123 t/s aggregate. The engine also supports DeepSeek V4 Flash, Qwen3.8 Flash-Next, Qwen3 ASR and TTS, Qwen Image 2.1, and MiniMax H3.