Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports 50.3 t/s generation and 1084 t/s prompt processing.
Source evidence (verbatim):
Source: https://omlx.ai/benchmarks/performance/ogokptif
# Qwen3.8-27B-oQ5e-mtp on M5 Max (40c) - oMLX Benchmark
Hardware: Apple M5 Max (40-core GPU), 128 GB unified memory. Engine: oMLX v0.7.1.dev1. Model: Qwen3.8-27B-oQ5e-mtp, 5-bit. Multi-token prediction (MTP) is enabled with adaptive max depth 3, so this is a speculative decoding run; DFlash and SpecPrefill are disabled. Single request. At 8k context: 1,084 tokens/s prompt processing and 50.3 tokens/s token generation. At 4k context: 1,137 prompt and 58.4 generation; at 1k: 895.8 prompt and 54.6 generation; at 16k: 882.8 prompt and 75.5 generation.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8 27B at 31 t/s on a single RX 7900 XTX 24GB with 32GB DDR5 6000 system RAM.
Setup is llama.cpp with Q4_K_M at 64K context.
User feels the setup is not optimized for coding on large projects and asks for resources explaining how quantization, KV cache types, context size, batch size, GPU offloading and expert hits affect memory use, speed and quality.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Strata running Qwen3.8 Flash-Next Coder at about 80 t/s on a single 100-word poem prompt, and about 80 t/s total with parallel=2 serving two prompts at once.
No hardware, quant or context length is stated.
The user asks what parallel=2 buys when total throughput stays flat, suggesting agents busy with tool calls would leave the running one full use of the engine.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B at 42.2 tok/s single-user on one DGX Spark, with prefill at 2,158 tok/s.
Setup is vLLM with unsloth's NVFP4 weights on the GB10's FP4 tensor cores, plus DFlash2 speculative decoding with 10 drafts from an NVFP4 drafter and a small vLLM plugin.
At 16k context it reaches 33.4 tok/s and at 100k 29.5 tok/s; the board's 16k two-user test gives 60.6 tok/s total, first among single-Spark entries. Quality checks against BF16 recover 99.3% on gsm8k, 101.1% on ifeval, 99.4% on humaneval and 101.2% on NarrativeQA.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User benchmarks Qwen3.8-Flash-Next on an RTX 5090 32 GB, reporting 155.3 t/s decode at 4K context with the Strata NVFP4 stack and 134.0 t/s with Infernix Dense8.
Setup is Strata 0.1.42-nvfp4.2 with an NVFP4/GPTQ pack, int8 KV cache, MTP speculative decoding (--spec 6), 15 CPU expert workers and 128 GB DDR5 at 6000 MT/s; one request at a time, 256 output tokens, greedy sampling.
At 32K Strata gives 154.7 t/s and Infernix 144.2 t/s; at 128K Infernix is faster at 147.3 t/s against 142.1 t/s. Infernix answers sooner (1.02 s vs 1.42 s at 4K). A retained Unsloth UD-Q4_K_XL run on the same Strata binary reached 130.5 t/s at 4K. The user notes the three stacks use different model representations and that this is not an intelligence ranking.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
agenticlong-context
User benchmarks Qwen3.8 27B (dense) at 9 t/s generation and 463 t/s prompt reading on a MINISFORUM MS-S1 MAX P495 (Ryzen AI Max+ PRO 495, 192GB) at empty context.
Setup is llama.cpp build 23f7aab with Vulkan, Q6_K_XL weights, q8_0 KV cache and flash attention, ubatch 512, all layers on the GPU.
With MTP speculative decoding (--spec-type draft-mtp) the same model reaches 24 t/s with 82.9% draft acceptance. At 32k context it reads at 286 t/s and writes at 9 t/s; at 128k, 131 t/s and 8 t/s. The user could not reproduce an f16 KV collapse at depth and says the machine is not worth $8k.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
agentictool-use
User benchmarks eight models on 33 DevOps agent tasks with taracode 3.3.1 on taraengine 0.3.1, at temperature 0 and 32k context, one run per model. On an RTX 5090 (32 GB), qwen3.8:27b leads at 97% pass rate and 126 tok/s, using 23.2 GiB of VRAM.
Setup is taraengine with Q4_K_M weights and MTP speculative decoding, which the engine pairs with seven of the eight models. The same catalog on an M4 Max (48 GB) runs 3.0 to 4.4 times slower, with qwen3.8:27b at 39 tok/s and 94% pass rate.
Against an earlier Ollama 0.35 board on the same 5090, gemma4:31b rose from 64 to 164 tok/s, gemma4:12b from 138 to 308 and qwen3.6:35b from 226 to 388, with every pass rate within four tasks. The user notes one run per row, rows within four tasks are a tie, and 33 tasks is small.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingtool-uselong-context
User reports Qwen3.8-Flash-Next at 87-98 t/s decode on short prompts on a single RTX 4090 24GB, after moving from 32GB to 64GB of DDR5-5800.
Setup is Strata 0.1.45 with Unsloth UD-IQ4_XS (about 55GiB of experts), 262K native context, q4_0 KV cache and MTP drafts at spec 4. 40.00 GiB of experts sit in RAM with no blob reads from the SATA SSD, leaving about 11GB free.
A 36K-token needle test gave 2,000 t/s prompt and 94 t/s decode; a 136K-token test gave 2,850 t/s prompt and 92 t/s decode. Expert cache hit rate is about 91%. The user notes IQ2_XS ran 140-160 t/s and IQ3_S 110-126 t/s on the same card, so the 4-bit move cost speed. Strata's quant table puts UD-IQ4_XS at 88.9% top-token match with Q8_0.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagenticlong-context
User reports Qwen3.8 Flash-Next at 37-39 tok/s writing and 856-895 tok/s reading the prompt at 60k context, split across five devices: an RTX 2060 Super PC, an RX 9060 XT PC, an M4 Pro MacBook with 24 GB, and two iPhone 17 Pro Maxes (A19 Pro).
Setup chains the devices over Ethernet and 10 Gb/s USB-C, with every weight held in VRAM, pinned RAM or unified memory and no disk streaming. Pipelined speculation from the model's own MTP draft head lands about 2.2 tokens per step.
At 16k context the user measured 42 tok/s writing and 728-744 tok/s prefill; at 32k, 39-41 tok/s and 844-852 tok/s. The agent session averaged 35 tok/s with bursts past 40. A second layout with 8-bit KV on the Mac and phones writes 31-33 tok/s at 60k and read one prompt past 164k tokens. PC 2's 16 GB caps prefill at about 980 tok/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8-Flash-Next at 72 t/s decode on an RTX 4090 with Strata, using Unsloth's UD-IQ4_XS quant at 262K context.
Setup is Strata with route tail skip disabled (STRATA_ROUTE_TAIL_SKIP=0) and RAM-budget mode; prompt speed was about 2,000 t/s at 36K and about 2,900 t/s at 136K.
The default skip of 7 gave 93 t/s at both 36K and 136K but doubled KL against Q8_0 (0.053 to 0.115), so the user chose quality. Strata's calibrator moved --spec-min-p from 0.5 to 0.7 for about +4% and confirmed --pcie-frac 0.20; values at 0.55 or higher cost 20-40%.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8 Flash-Next running natively in NVFP4 under Basalt on an RTX 5090 at 400W with 128 GB of DDR5 RAM, giving 304 tok/s on structured output and 183 tok/s on prose.
Setup is Basalt with a .basalt pack that keeps stock NVIDIA experts and stores the dense layers as Q6_K; adding a peer RTX 5060 Ti raises the figures to 479 tok/s structured and 239 tok/s prose. Prefill reaches 8,582 tok/s on a 64K prompt, 7.6s to first token.
Against Infernix on the same NVFP4 weights and box the user claims up to 1.3x on decode and 1.6x on prefill. Quality versus Q8_0 over 100 chunks: PPL 3.308 (Q8_0 3.286), KLD 0.068, 91.5% same top-1.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports running Qwen3.8 Flash-Next at about 28 tokens per second, with most of the output going into thinking.
No hardware, engine, quant or context length is stated.
The user hopes Qwen 4 will think less at the same or higher intelligence, and notes no exact release date has been announced.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports 29.8 t/s generation and 1287 t/s prompt processing.
Source evidence (verbatim):
Source: https://akougkas.io/journal/2026/serving-qwen38-27b-on-a-radeon-ai-pro-r9700/
# Serving Qwen3.8-27B on a Radeon AI PRO R9700 is not a one-liner
Hardware: Radeon AI PRO R9700 (Navi 48, gfx1201) with 32 GB VRAM and 640 GB/s, Ryzen 7 5700X3D (8 cores), Ubuntu 24.04.4, kernel 6.17 OEM. Engine: llama.cpp b10687 (commit c841aee), HIP build on ROCm 7.2.4. Model: Qwen3.8-27B, Unsloth IQ4_NL (15.21 GiB). Single-stream llama-bench with no speculative decoding: -ngl 99 -fa 1 -ctk q8_0 -ctv q8_0 -p 2048 -n 512 gave 1287 tokens/s prefill (pp2048) and 29.8 tokens/s decode (tg512). The same bench for other quants: UD-Q4_K_XL 1151 / 26.7, UD-Q5_K_M 946 / 24.5, UD-Q6_K 821 / 23.2 (prefill / decode tokens/s).
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen 3.8 27B at 42.7 t/s generation and 658 t/s prefill at 20k context on a single RX 9060 XT 16GB.
Setup is a fork of llama.cpp (achyudh/llama.cpp) with an unsloth IQ4_XS GGUF, Q4 KV values with Q8 keys, MTP plus n-gram prediction, and KV cache streaming; 24GB of DDR5 system RAM, 16GB of it used for this model.
At 200k context the user reports 35 t/s generation and 300 t/s prefill, with a 262k context target. The user notes the build needed PCIe atomics fixes.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
agenticcoding
User reports Qwen3.8 Flash-Next at around 50-60 t/s decode with 1300 t/s prefill on a dual RX 7900 XTX system.
Setup is Strata with a Q3_K_S GGUF and Q8 KV cache, running on the rest of the system while a Qwen3.8 27B Q4_K_XL model with 131k context and Q8 KV sits on the second card behind the chipset at gen3 x4.
The 27B gets around 40 t/s decode and 700-800 t/s prefill. An unused flag raises Flash-Next prefill to about 2300 t/s but makes inference unstable, and a layer split keeps decode the same while bringing prefill near 3000 t/s. User is experimenting with pi/hermes agent setups and is not satisfied with how the two models work together.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
long-context
User reports 30 t/s generation and 864 t/s prompt processing.
Source evidence (verbatim):
Source: https://gist.github.com/ryan4yin/48617bbddacc7067f10799770b7cc33f
# Qwen3.8-Flash-Next on an RTX 4090 with llama.cpp (mainline), 96GB DDR5
Hardware: RTX 4090 24GB, Intel Core Ultra 7 270K Plus (24C/24T), 96GB DDR5-5600 (2x48GB dual channel). Engine: mainline llama.cpp, image ghcr.io/ggml-org/llama.cpp:full-cuda13. Model: unsloth/Qwen3.8-Flash-Next-GGUF UD-IQ3_XXS (about 82GB) plus mmproj-F16. Flags: --ctx-size 120000 --fit on --fit-target 256 -t 20 -tb 24 -b 6144 -ub 6144 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 -np 1. Experts are offloaded to system RAM. No speculative decoding or MTP (llama.cpp does not support the MTP head yet). Measured with this llama.cpp config: decode about 30 t/s near 6K context, 22.2 t/s at 90K context, 19.5 t/s at 110K input; prefill 864 t/s at batch 6144 (762 t/s at batch 2048).
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
agenticcodingtool-use
User reports about 83 tok/s generation on an RTX 3090 24 GB running a Swift-1.5 fine-tune of Qwen3.8 27B at IQ3_S GGUF with MTP (draft 4) and vision on.
Setup is Ollama 0.35.1 on Windows 11, one request at a time, 131072-token window fully on the card.
The figure is for a local agent driving ten Windows apps; the user notes writing files directly is still about 10x faster and made the agent stop doing that by default.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User asks which inference engine people prefer on Windows, Strata or Rulith, on a Strix Halo.
They run Rulith with an IQ4 quant of Qwen3.8 Flash-Next Uncensored and get around 50 to 60 t/s.
They have not tried Strata yet and are asking what others use on Windows.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8-27B at 53 tok/s mean decode on an Intel Arc Pro B60 24GB, with a 64K context window.
Setup is llama.cpp on the Vulkan backend with f16 KV cache, -ub 256 and a q8 draft KV cache, using DFlash2 speculative decoding and Intel's FA prefill kernel. Driver 32.0.101.8976 on Windows 11.
The user notes smaller cards (B580/B570/B50/A770) will not fit this 27B, though the Vulkan tuning applies. The same card also runs LTX-2.5 video generation via ComfyUI on XPU.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codinglong-context
User reports WHIRL 0.2.0 running Swift-1.5 27B MXFP4 on the Radeon 8060S (Ryzen AI Max+ 395, ASUS ROG Flow Z13 at 80 W) at 34.2 t/s decode on a coding chat and 470.8 t/s prefill on an 8k-token prompt.
Setup is the WHIRL engine in its default balance mode with MXFP4 GGUF weights, one user, greedy decoding, MTP speculation on, compared against llama.cpp b11214 ROCm and Vulkan builds.
WHIRL leads llama.cpp from 2k to ~250k tokens; the smallest lead is dense decode on the 8060S, where 34.2 t/s beats 28.3 (Vulkan) and 27.6 (ROCm). A cold coding-agent session took 95.7 s versus 145.6 s on ROCm and 264.4 s on Vulkan. The same post also benchmarks Ornith-1.5-35B-A3B MXFP4 and a Radeon AI PRO R9700, and notes llama.cpp decodes Ornith without speculation while WHIRL uses MTP, so those rows are not like for like.