Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
long-context
User reports Qwen3 30B-A3B at 58.57 t/s decode and 920 t/s prefill on an AMD Strix Halo 128GB at 12,001 tokens of context.
Setup is llama.cpp with the Q4_K_M GGUF over Vulkan, 49,152-token context, 96 GB of the 128 GB LPDDR5X allocated to the iGPU.
At 48,001 tokens prefill drops to 254 t/s and decode to 30.06 t/s; at 11 tokens prefill is 122 t/s and decode 88.40 t/s. ROCm gives about 28 percent better prefill while Vulkan gives about 13 percent better decode.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8 Flash-Next (QFN) running on an AMD Strix Halo 128GB box, with 30-40 t/s generation and 900-1000 t/s prefill in real-world use over several days.
Setup uses the Halogen engine, which the user praises as the best current setup for this hardware and model.
The user compares QFN to a 27B model and finds them similar in capability, feeling that 27B already saturates their personal use cases.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8-Flash-Next at 47.1 t/s on an AMD Strix Halo 128GB, up from 16.8 t/s baseline, for file rewrite at about 8k context.
Setup is llama.cpp on ROCm 7.1 with the UD-IQ4_XS GGUF, using GPU TOP_K improvements, native MTP speculative decoding and ngram-mod augmentation.
New code at 8k context reached 31.7 t/s; at about 24k context, file rewrite reached 28.6 t/s and new code 25.4 t/s, against baselines of about 15 t/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.6 35B-A3B at ~50 t/s on a 128GB Strix Halo APU running Nobara Linux.
Setup is llama.cpp with the Q6 quant.
User asks what newer models are recommended for the 128GB Strix Halo, either faster with the same capabilities or better with the same speed.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
long-contextsummarizationcreative-writing
User reports Qwen3.8 27B at roughly 5–6 t/s on Strix Halo with max GPU offload.
Setup is Windows + WSL2 + Docker, LM Studio with the Q4_K_M GGUF, Open WebUI frontend, and 262K context with context compaction enabled.
The user calls the experience poor: refining a 12-slide deck repeatedly stalled for minutes and returned nothing across 3–4 retries, and asks whether the model, the 262K context, LM Studio, Open WebUI, or the stack is at fault.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
agentictool-use
User reports Qwen3.8 NVFP4 at 20.6 t/s on an AMD Strix Halo 128GB, with 17.3 t/s on Windows.
Setup is Atlas Inference running natively on HIP with custom DP4A math on gfx1151, using K=4 draft speculative decoding and a 65k agentic context window. The engine is the sole cross-architecture submission for MLPerf Inference v6.1 Edge Agentic LLM SingleStream, scoring 87.76 normalized BFCL and 84 on HA-20.
Median TPOT is 47.3 ms and median TTFT is 2.7s across 65k context windows.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingmultilinguallong-context
User reports DeepSeek V4.1 Flash Q2 (~340 GiB MoE) running on a single AMD Strix Halo 128GB box with 124 GiB unified memory, reaching a highest completion-token rate of 7.12 tok/s including reasoning.
Setup is the kyuz0 ds4 ROCm runtime with SSD expert streaming (--ssd-streaming-cache-experts), ROCm 7.2.4, 8k context, and a host-side patch. Small coding and Czech tests completed at request-wall medians of 29.2 s (Python), 26.4 s (bugfix), and 20.4 s (Czech); 64k prompts pass but 128k never produced a final answer in a 905 s bounded calibration.
User stopped the experiment, calling it not a practical winner. A Three.js coding test produced 20,575 output tokens over ~83 minutes at ~3.9-4.2 end-to-end tok/s with zero accepted games. The same box measured Qwen3.8 Flash-Next at 44.66 tok/s short, 41.37 @ ~32k, 40.08 @ ~64k, and 38.03 @ ~126k with MTP decode, and GLM-5.3-Flash at 14.63 tok/s on ROCm versus 8.57 tok/s on Vulkan.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User benchmarks 12 quantizations of Qwen3.8-Flash-Next on an AMD Strix Halo 128GB, reporting 24 t/s decode and 312-382 t/s prefill at 512-2K tokens with llama-server.
Setup is llama.cpp (llama-flash-next fork) with the Heretic2 IQ4_XS NGQ4 GGUF at 65,536 context, using Vulkan backend; the 92 GB model fits in 128 GB unified memory.
Halogen Flash Server shows 2-4x faster prefill (453-1,203 t/s) but accuracy collapses to 65.2% without its proprietary overlay, and it forces thinking mode.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8 Flash-Next at 38.3 t/s decode and 937 t/s prefill on a Ryzen AI Max+ 395 with 128 GB, at 1,004,581 tokens of context.
Setup is the halogen server (0.12.0) with HALOGEN_ROPE_YARN=4 and HALOGEN_CTX=1048576, greedy, 64 tokens, using the default speculative drafter; the 1M row is a single cold request.
At 258,794 tokens the same setup gives 45.0 t/s decode and 1,114 t/s prefill; a follow-up turn over the prompt cache at 1M reaches first token in about 0.55 s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.6 35B-A3B at 77.92 t/s with MTP on an AMD Strix Halo 128GB, versus 53.83 t/s without MTP.
Setup is llama.cpp build b11009 with the unsloth UD-Q4_K_XL GGUF at 4096 context, using the Vulkan backend, flash attention, and draft-mtp speculative decoding with spec-draft-n-max 2.
The benchmark also covered Nemotron-3.5-Lightning-30B-A3B NVFP4, Qwen3.8 27B OBLITERATED Q4_K_M, GLM-4.5-Air Q4_K_M, Qwen3.8 27B UD-Q4_K_XL, Qwen3.8 Flash-Next UD-IQ4_XS, and Qwen3.6 35B-A3B BF16.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingcreative-writing
User reports Qwen3.8 27B Flash-Next at 28 t/s generation and around 250 t/s prefill on a Strix Halo 128GB at a 131k context limit.
Setup uses the UD_Q2_KD_XL GGUF quant. The model was used for web game development with an MCP for handheld console input standardization and the Blender MCP for model creation.
User compares it against Gemini Flash 3.8 on an identical one-shot backrooms game prompt; Gemini finished in about 5 minutes but produced a low-effort result, while Flash-Next took 2 days including QA and play testing and produced a much more detailed and immersive game, though input was inversed. User says it feels like Opus 4.8 at home and wants a Qwen 35B MoE with engrams at 40 t/s decode and 262k context to go fully local.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8 Flash-Next at 52 t/s decode and 1300 t/s prefill on Strix Halo.
Setup is forked llama.cpp builds and halogen-flash-server, described as roughly double decode and 5-6x prefill performance over prior Strix Halo results.
User offers a community reflection on local LLM optimization rather than a detailed benchmark report.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8 Flash Next reaching 1.2k t/s prefill on Strix Halo 128GB.
Setup is a custom llama.cpp branch with a custom HIP runtime, built to match the closed-source Halogen server's prefill numbers; the community fork had only reached 400 t/s.
The user plans to clean up the work and submit PRs to mainline and the community fork, noting the approach may also help the GLM 5.3 Flash architecture.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codinglong-context
User benchmarks five Strix Halo llama.cpp forks with Qwen3.8 Flash-Next on a Flow Z13 (Ryzen AI Max+ 395, 128 GB, gfx1151).
halogen 0.5.4 at 70W reaches prefill 1192/1350/1315 t/s at 8k/32k/131k and decode 33.2 t/s serial and 43.1 t/s drafted. Nathan's toolbox v0.7.4.1 on Vulkan reaches 26 t/s serial and 34.3 t/s drafted, with prefill 539/433 t/s at 8k/32k. myhacsint reaches 25.7 t/s serial and 35.6 t/s drafted, with prefill 532/584/538/427 t/s. strix-llama was not run; the thread reports 14 t/s serial and prefill 532/397 t/s.
Speculative decoding fails the byte-identity gate on every GGUF build tested, with 3/5 to 9/10 prompts differing at temp 0. Serial reruns are byte-identical 10/10. The halogen 0.5.3 changelog claims 1246/1424/1358 t/s prefill at 8k/32k/131k. The model is Qwen3.8 Flash-Next; parameter count is not stated.
User reports Qwen3.8-Flash-Next-UD-Q5_K_XL on Strix Halo 128GB with mmproj, MTP, a 64000 Q8 KV cache and 2GB cache-ram, leaving only 1-2GB free memory.
Q4_K_XL (77GB) and Q5_K_XL (108GB) were also tested. MTP adds 5.5GB even when the file is 2.8GB. F16 KV cache uses 1GB per 10000 tokens; Q8 uses 1GB per 20000 tokens.
Perplexity results: Q5_K_XL Top-1% 94.795, Mean KLD 0.018175, 99.9% KLD 0.591240; Q4_K_XL+50GB-NGRAM Top-1% 93.786, Mean KLD 0.027853, 99.9% KLD 0.948222; Q4_K_XL+25GB-NGRAM Top-1% 93.285, Mean KLD 0.033378, 99.9% KLD 1.090875. The F16 base-kld could not be run due to insufficient memory.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingagenticlong-context
User compares Qwen 3.8 27B at Q6_K and about 31 GiB resident against Qwen 3.8 Flash Next at UD-Q4_K_XL and about 86 GiB on an ASUS ROG Flow Z13 with Ryzen AI Max+ 395, 128 GB unified memory, running Arch Linux. Decode is 10-15 tok/s.
Setup is the llama.cpp backend with the LlamaStash tool and the Pi coding harness. Cold prefill runs 31k tokens in 3 min (about 172 t/s) and a 128k window in about 18 min (about 118 t/s). Warm follow-up turns take about 45s. MTP gives 7.3 to 22.4 tok/s on an empty window and 1.15x at the full 256k.
Flash Next completes the same 5/5 coding tasks with 45% fewer tokens, 76.5s against 289.8s for 27B. On the Artificial Analysis index Flash Next scores 40 against Opus 4.8 at 42, and 27B xhigh scores 34 against Opus 4.6 at 32. Thinking is 90-95% of generated tokens. The setup costs $0/month and runs fully offline.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
visionlong-context
User benchmarks Qwen3.8-Flash-Next-Uncensored on a Bosgame M5 (128GB/2TB) running Fedora 44 with Vulkan driver 26.1.7, reaching 198.99 t/s prompt processing and 21.53 t/s generation at 200,000 context.
Setup is a llama.cpp fork (strix-halo-qwen4exp-b10685) with the Q5_K_M quant and a shared-Q8_0 MTP draft model.
llama-benchy results across context lengths: pp2048 460.38 t/s, tg512 34.86 t/s; pp4096 465.96 t/s, tg512 33.04 t/s; pp8192 437.14 t/s, tg512 31.18 t/s; pp16384 440.89 t/s, tg512 31.21 t/s; pp32768 407.48 t/s, tg512 29.65 t/s; pp65536 351.55 t/s, tg512 27.40 t/s; pp131072 265.54 t/s, tg512 23.87 t/s; pp200000 198.99 t/s, tg512 21.53 t/s. MTP acceptance rate stays at 80% even at context above 200K.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagenticlong-context
User reports Qwen3-Coder-Next UD-Q6_K_XL at 36.75 t/s on tg32 on an AMD Strix Halo box, with prefill at 545.77 t/s on pp2048.
Setup is llama-server with -ngl 999, flash attention on, -c 262144, f16 KV cache and one slot, measured with llama-benchy 0.4.1 in API latency mode.
The tg32 rate is 36.75 ± 0.05 t/s with a peak of 37.94 ± 0.06 t/s. A 27B 8-bit XL model was too slow to be workable.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User benchmarks Nex-N2.5-mini in ROCmFP4 GGUF on AMD Strix Halo against Qwen3.8-27B, decoding at 76.9 t/s versus 14-34 t/s.
Setup is the HaloFPX engine with weights from julianmb/Nex-N2.5-mini-ROCmFP4-GGUF.
Nex-N2.5-mini scores 73.4 on Terminal-Bench 2.1 against 73.0, 63.4 on WebArena against 64.8, and 43.8 on SWE-Bench against 61.7.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports running 3 local sessions of GLM 4.7 at about 40 tok/s each on an AMD Strix Halo mini PC with 128 GB unified RAM.
The model family is inferred as GLM-4.7 from the title and text.
The user is positive about the setup for budget local AI.