Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagentic
User reports Qwen3.8-Flash-Next prefill at 873 t/s on a 4K prompt on Strix Halo 128GB (Ryzen AI Max+ 395, 96 GB dedicated to the GPU) under native Windows 11.
Setup is a Windows port of the Gufo engine on one Strix Halo, UD-Q4_K_XL weights with a Q8_0 MTP head, MTP speculative decoding with survival-based draft stopping and a pruned Latin/code draft vocabulary, 256K context allocated.
Decode is reported separately for a summary prompt and a code prompt with no overall figure: 45.3 and 39.7 t/s at empty context, 40.9 and 39.7 at 4K, falling to 38.4 and 37.1 at 158-174K. Prefill stays between 871 and 930 t/s up to 174K. A follow-up turn on a 158K conversation reached first token in 0.4 s from the prompt cache, then 38.6 t/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 Flash-Next decoding at about 45 t/s on a 128GB Strix Halo.
Setup is Halogen 0.17.2; the user says decode stays consistent at that rate even at high context.
The user calls it crazy good for a 177ish billion model and praises the engine's rapid development.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports running Qwen3.8-Flash-Next UD-Q4_K_XL at 262K context on a Strix Halo 128GB (Ryzen AI Max+ 395) with a 96GB GPU carve-out, using Gufo (a Windows-port fork) with Unsloth Studio as client.
Setup uses a 64GB minimum / 192GB maximum page file and a source patch to Gufo's snapshot budget (set to 16GB via environment variable) to fix a Windows memory misread that capped the cache at ~264 MiB. Disk cache is off.
Performance is about 900 to 1,000 tok/s prompt processing and 35 to 55 tok/s decode depending on context length. First token at 120K+ tokens now arrives in 2 to 5 s instead of 40 s or more. User is happy with the setup.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagenticlong-context
User reports Qwen3.8-Flash-Next at 38 t/s serial decode and 1,311 t/s prefill on a Strix Halo 128GB (Ryzen AI Max+ 395, Radeon 8060S) box.
Setup is halogen 0.17.1 with a 262k context, MTP speculative decoding on, and the model fully in unified memory; the same run reaches 64 t/s with MTP and 72.7 t/s aggregate across two concurrent streams.
A llama.cpp ROCm build with qwen4exp MTP reached 47 t/s, and halogen 0.16.2 gave 42-46 t/s with NPU calls 30x slower. The NPU sidecar handles search, dedup, decisions and screening; a keyword-poor task averaged 13.6 min with NPU retrieval against 18.7 min without.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
agenticcodingtool-use
User reports MiMo V2.6 Flash MOPD mixed GGUF at 23.59 t/s decode and 37.54 t/s prefill on an AMD Strix Halo 128GB.
Setup is llama.cpp (Vulkan) with MQ-IQ2-XXS-XS-Q8 GGUF and q8_0 KV cache, 256K context, thinking disabled, one request at a time.
Longer inputs slow decode to 15.89 t/s at 4,860 tokens and 7.39 t/s at 48,600 tokens. MTP was disabled; a preliminary test showed MTP 3 taking 225.591 s versus 39.896 s without it. An agent benchmark scored 330/900 (36.7%) with all ten tasks timing out.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.5-35B-A3B at ~45 t/s single user on a Strix Halo (Radeon 8060S, gfx1151) with 90112 MiB unified memory visible to the GPU.
Setup is llama.cpp HIP build (commit 4d99d45) with Q4_K_M GGUF, 131072 context, -ngl 999, flash attention on, and -dio, running in a Proxmox LXC with ROCm 7.2.0.
Aggregate throughput reaches ~168 t/s at 16 parallel slots, where the user notes memory bandwidth becomes the limit.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8-Flash-Next at 1,400 t/s prefill and 46 t/s decode on a Strix Halo 128GB laptop at 70 W.
Setup is Gufo as the engine, with the model running at xhigh effort. The user notes the model ran on an older Halogen version (0.14.0) and Halogen dropped the connection once, so the last part ran on Gufo.
The user compares the local model against Claude Opus 5.5 on a coding task, finding the local model's PR better in tests and edge cases, though Opus was 2 to 10 times faster overall.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next at 53.8 t/s output and 1293 t/s prompt processing on Strix Halo 128GB.
Setup is Strata with UD-IQ4_XS weights at 8K context; the release adds official Strix Halo support, marked experimental and Linux only.
At 64K the run gives 46.6 t/s output and 1370 t/s prompt processing, and at 128K 51.4 t/s output and 1320 t/s prompt processing. The user says it can go up to 1M context length without big speed loss.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 Flash-Next at about 50 tok/s decode with MTP on a Strix Halo 128GB box, even at 116k context.
Setup is a forked Gufo engine with the Unsloth UD-IQ4_XS quant (about 89GB) and an MTP sidecar, running in the 120W performance profile.
Prefill is 1500+ tok/s from about 3k tokens up, peaking around 1570; on HumanEval prompts decode is about 76 tok/s with MTP versus 28 without. The balanced profile is around 10% less prefill, and very short prompts are slower (about 1350 at 2k) due to a fixed cost per request.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B at 33.96 t/s generation on Strix Halo 128GB.
Setup is Gufo with Q4_K_XL GGUF and DFlash2 speculative decoding using a Q4_K_M draft model. The prompt processing figure of 183.3 t/s is noted as low because of the short prompt.
User notes it is slower on Q8_0 but still faster than llama.cpp, and is testing the new AUR package.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
vision
User reports Ornith-1.5-35B-A3B at 72.6 t/s generation and 1504.9 t/s prefill on AMD Strix Halo 128GB.
Setup is llama.cpp-rocm with Q4_0_ROCMFP4 GGUF quantization, -ngl 99, benchmarked with llama-bench -p 256,2048 -n 256 -r 5.
Seven ROCmFPX quant tiers are provided; the Q4_0_ROCMFP4_FAST tier is the bare baseline. The model is a 35B MoE with ~3B active parameters and supports vision via bundled mmproj and MTP speculative decoding.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8 Flash at 1200+ t/s prefill and 40 t/s decode on a Strix Halo laptop.
Setup is a Strix Halo with 128GB unified memory; engine, quant and context length are not stated.
User says the model handles their pull requests and rates its quality as Opus 4.8 level, also mentioning some 27B runs.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks Qwen3.8-Flash-Next on a Bosgame M5 (Strix Halo 128GB) across three backends. Gufo v0.7.0 with unsloth UD-Q4_K_XL and MTP reaches 54.0 t/s decode on code, 53.6 t/s on JSON, 29.5 t/s on prose, and 32.9 t/s at 19k context, with prefill 1124/1183/1179 t/s at 4k/19k/38k. strix-llama (ROCm) with ISTA GSQ-RCO IQ3_S and MTP gets 37.9/36.0/24.4/26.6 t/s decode and 734/798/794 t/s prefill; mainline llama.cpp (Vulkan) without MTP gets 27.5/27.5/27.3/25.5 t/s decode and 329/328/285 t/s prefill. GPU memory at 128k context is ~85 GiB for Gufo, ~62 GiB for strix-llama, ~57 GiB for mainline. The user notes Gufo is fastest but only accepts unsloth Q4_K_XL, while IQ3_S on strix-llama leaves room for Gemma 4 26B-A4B alongside. MTP did not work on mainline Vulkan. llama-bench on strix-llama with IQ3_S and no MTP gave pp2048 944 t/s (845 at 32k) and tg128 26.4.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports GLM-5.3-Flash at 580 tok/s prefill (3.5K) and 546 tok/s at 64K on a single Strix Halo 128GB mini PC (Ryzen AI Max+ 395, gfx1151).
Setup is the Kyojin engine built on ExLlamaV3 with EXL3 weights, 99.7 GB pack, MTP speculative decoding with 2 draft tokens.
Decode is 26 to 30 tok/s with MTP. A second model, MiMo-V2.6-Flash-MOPD (105 GB), reaches about 650 tok/s prefill at 4K and 32 prose / 35 chat / 44 code tok/s with speculation, 29 plain. KLD vs official FP8 is 0.151 for GLM and 0.0713 for MiMo. GLM at 128K context and other GPUs are not measured yet.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
agentic
User reports Qwen3.8 Flash-Next at 40 t/s average on agentic tasks on Strix Halo 128GB.
Setup is Gufo with UD_Q4_XL quant, VRAM set to 96GB.
User notes the speed holds at higher context lengths and recommends setting VRAM to 96GB on Windows.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B FP4 at 14.0 t/s bare decode on an AMD Strix Halo (Radeon 8060S), with prefill at 440 t/s on pp512.
Setup is a llama.cpp fork with a Vulkan backend on stock Mesa RADV 26.0.8, greedy, 300 tokens, with no speculative decoding.
With an FP4 DFlash2 sidecar and adaptive drafting (n_min 3, n_max 7) the same model reaches 65.6 t/s on structured output and 26.1 on prose, on a short high-power burst (79 °C, 115 W). On the everyday power profile that configuration reads 55.0 t/s on structured output.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
coding
User reports Qwen 3.8 Flash Next at 66.55 tok/s decode on a Ryzen AI Max+ 395 / 8060S Strix Halo 128GB under WSL2 on Windows.
Setup uses a 260k-token context; prefill measured at 790.92 tok/s in a separate shorter-output run. Two full 260k sessions ran at 25.45 tok/s each, 50.90 combined, with no aggregate speedup over solo.
Windows idle usage is 9-12 GB including Codex; the two-session run left at least 18.86 GB available. Three full contexts froze the PC.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next (125B-A6B) at 41.0 tok/s decode on short context and 1183 t/s prefill on a 95.6K-token prompt, on a Ryzen AI Max+ 395 with 128 GB (Strix Halo).
Setup is a patched llama.cpp HIP runtime with Unsloth UD-IQ4_XS weights (93.7 GB), MTP speculative decoding with three draft tokens, and optional Q8_0 K/V cache; the GPU carve is set to 96 GB.
Decode falls to 32.6 tok/s at 86K context; three and four concurrent conversations reach 55.7 and 60.4 tok/s summed. Prefill is given as a three-run range of 1167-1191 t/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks Qwen3.6-35B-A3B at 50.8 t/s decode and 2384 t/s prefill on a Strix Halo 128GB using the gufo engine with Q8_K_XL quantization.
Setup is gufo with Q8_K_XL GGUF and no KV cache quantization, running on the Ryzen AI Max+ 395 with 128 GB unified memory. The user compares gufo against llama.cpp across multiple quantizations and speculative decoding modes.
Gufo achieves higher prefill speeds (up to 2702 t/s on Q6_K_XL) but llama.cpp decodes faster on Q6_K_XL by 3% to 9%. With DFlash2 speculative decoding at 7 draft tokens, gufo reaches 78.6 t/s on Q8_K_XL versus llama.cpp's 51.7 t/s. The user notes that speculative output is not bit-identical to plain greedy output on either engine.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8 Flash-Next at 20 t/s generation on a Strix Halo 128GB laptop with 64GB RAM.
Setup is llama.cpp via lemonade with Vulkan on Windows, GSQ-RCO GGUF quant, 16k context, Q4 KV cache, and lazy load enabled. The model uses about 30GB of GPU memory but also consumes roughly 20GB of CPU memory, which the user suspects is the n-gram table not staying on SSD.
The user notes the model loaded and worked rather well, but a default prompt ran out of the 16k context due to extended thinking.