Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next at 1,405 t/s prefill and 72.6 t/s decode at 256K context on a Bosgame M5 mini PC running Windows 11.
Setup is a Windows port of Gufo, patched with improvements from the upstream Linux repository; the user says each port was benchmarked and regression-tested.
Peak figures were 1,601 t/s prefill and 75.4 t/s decode with MTP speculation enabled.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingtool-usemathlong-context
User benchmarks nine local Qwen 3.8 configurations against Claude Opus 4.6, with Flash-Next IQ3_S split across both GPUs reaching 81.5 t/s decode at 96K context and 75.5 t/s on a short code prompt.
Setup is Strata with IQ3_S GGUF weights (83.6 GB) on an RTX 5060 Ti 16GB over PCIe x16 plus an RTX PRO 4000 Blackwell 24GB over PCIe x1, with experts streamed over PCIe; the 27B IQ3_S runs at 93.2 t/s on a short prompt and 48.5 t/s at 32K.
The narrow x1 link is a major bottleneck: Flash-Next read a 4K prompt at about 145 t/s on the PRO 4000 alone versus 1,433 t/s on the 5060 Ti. Best local official-sampling HumanEval was 96.3% against 100% for Opus, while local models won MATH-500 (96.5% vs 95.0%) and instruction following (100% vs 94.9%).
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next running at 200-300 t/s on a single RTX 5090, entirely in the card's 32 GB with no offloaded experts.
The weights are a 1.66bpw EXL3 quant; cache quantization is used, and the user notes one 32 GB GPU leaves little room for context.
The speed is given only as a range. The user calls the sub-2bpw quant stable, with no reasoning loops.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User benchmarks Kapitsa 0.2 with its KFlash speculative decoding at 63 tok/s on Qwen3.8 27B on a MacBook Pro M4 Pro 48GB.
Setup is Kapitsa with 4-bit weights, decoding a 512-token coding answer at temperature 0.6, median of 5 runs.
The post also cites Splash 1.4.0 (DFlash2) at about 50 tok/s and Kapitsa at about 65 tok/s in its title, and mentions an M1 Pro 16GB MacBook Pro and Qwen3.8 4B/35B runs that are not shown in the excerpt.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 Flash-Next at around 40-45 tok/s on an RX 6900 XT 16GB, depending on the prompt.
Setup is Strata with a Q2 quant on a 32GB DDR4 system (Ryzen 7 5700X, B450 board).
User asks whether to upgrade to 64GB RAM with an X570 board or add a second 6800/6900 XT for 16GB more VRAM, noting the current board's second slot is PCIe 2.0 x4. A dual-GPU goal would be Qwen3.8 27B at Q4 or higher.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User benchmarks Qwen3.8-Flash-Next-W4A16 at 87.7 t/s median per-request decode on 4x Intel Arc Pro B65 (128GB total VRAM).
Setup is vLLM 0.30.0 with INT4 W4A16 weights at zero context, single request; the table also gives total throughput and medians at C2 and C4 and at contexts up to 200K.
Measurements are sustained-decode medians from 30-second tests, not totals divided by concurrency. A second Flash-Next run at the same settings reached 43.4 t/s at C1. The post also sweeps Qwen3.8-27B on 1x and 4x B65 in INT4, FP8 and BF16, and mentions starting layer offload to system RAM with Flash-Next.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagentic
User benchmarks Qwen3.8-27B (Q4_K_XL GGUF, 17.6 GB) at 40.9 tok/s decode on an RTX 3090 against a Mac mini M5 Pro 64GB at 15.6 tok/s.
Setup is llama.cpp with q8_0 KV cache, 131072 context, 256-token completions, prompts of about 8,300 tokens. Prefill was 1,150 to 1,210 tok/s on the 3090 and 322 to 339 tok/s on the Mac.
On a real agent task the 3090 finished in 59 min 51 s against the Mac's 3 h 7 min, writing 27.3 against 10.4 output tokens per second. MLX with nvfp4 weights decoded at 29.6 tok/s on the Mac, and MTP lifted the Mac's llama.cpp decode from 15.6 to 25.6 tok/s. The user calls the ratios rough, with one task and one run per machine.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8 Flash-Next 125B-A6B at around 65 tok/s with Strata on unspecified consumer hardware.
Setup is Strata with an IQ3_XXS quant; no GPU, context length or KV cache is named.
The user calls the speed amazing but says coding quality, software design and code review still feel like compromises next to cloud APIs.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8 Flash-Next Swift-1.5 at 110-120 tok/s decode and 2000-2500 tok/s prefill on an RTX 3090 with Strata.
Setup is Strata with IQ3_S at 262144 context on the 24GB card in a 128GB DDR4 system.
The same model on an RTX 4060 office PC with IQ2_XS and Q4 KV cache gives around 37-38 tok/s decode and 250 tok/s prefill. On an RTX 5060 with 32GB RAM, Strata prefill was around 100 tok/s. The user says hand-tuned llama.cpp only reached 30-40 tok/s decode, while Strata with the tiel variant gives at least 60 tok/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingagentic
User reports running Qwen3.8-27B-pi at 30-40 tok/s generation on an RTX 3080 10GB plus RTX 3060 12GB, with 32GB DDR5 4800 system RAM.
Setup is llama.cpp (llama-server) with Q4_K_M weights, q8_0 K cache and q4_0 V cache, 75776 context, layer split with tensor-split 107,93, and MTP draft speculation (draft model Q4_0 on CUDA1, spec-draft-n-max 3).
Average MTP acceptance is about 0.75; pushing n-max above 3 causes OOM. User tried DFlash2 but found it slower than MTP, and is generally happy with quality and speed, having started at 10 tokens per second with Qwen 3.8.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8-27B Q4_K_M at a median of about 38 tokens/s on an RTX 3090 in a USB4 eGPU enclosure attached to a Beelink GTR7 mini PC.
Setup is llama.cpp with Q4_K_M weights, q8_0 KV cache, 64K context, flash attention on, and n-gram speculative decoding enabled.
Prompt evaluation runs about 900-1200 tokens/s on large prompts, with spikes up to about 280 t/s when speculation hits on repetitive code. VRAM use is 20.7 of 24.6 GB with the full 64K context.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
long-context
User benchmarks Qwen3.8-27B at 50.9 t/s per request (50.4 t/s total) at concurrency 1 on a 4x Intel B65 rig.
Setup is vLLM 0.31.0+xpu.llmscaler with INT4 weights; figures are sustained-decode medians from 30-second tests.
Long-context sweep: per-request decode falls from 50.4 t/s at 8K to 42.5 t/s at 128K and 38.9 t/s at 200K. Aggregate throughput rises to 468.2 t/s at C8 and 458.1 t/s at C16, with per-request rates of 62.1 and 30.1 t/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports an engine built with the crit toolkit running Qwen3.8-27B at 216 tok/s decode and 8,917 tok/s prefill on an RTX 5090.
Setup is a from-scratch engine generated by Claude Opus 5.5 from the crit repository, with 262K context and +0.3% perplexity versus the reference.
The build took 3.5 hours with almost no intervention, and the user says it performs in the same league as dedicated engines like NInfer.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
visionlong-context
User reports 50.44 t/s generation and 815.64 t/s prompt processing.
Source evidence (verbatim):
Source: https://hackernoon.com/how-i-got-qwen38-27b-running-at-50-toks-with-256k-context
# How I got Qwen3.8 27B running at 50 tok/s with 256K context
Two GPUs: an NVIDIA RTX PRO 4000 Blackwell SFF 24 GB (432 GB/s) holds the target model, the MTP head and the 256K KV cache, and an RTX 2000 Ada 16 GB holds the F16 vision projector. Debian 13, CUDA 12.9. Custom llama.cpp build with pinned PRs. Model Qwen3.8 27B, custom 5.01 bits-per-weight iMatrix/NVFP4 hybrid GGUF (16.3 GB), context 262144, Q4_0 KV cache. Target-only at about 40K context: 22.40 tokens/s generation. With MTP (n_max 3) at about 40K context: 44.95 tokens/s. Production ten-run mean with MTP n=8 plus n-gram: 50.44 tokens/s. Greedy target-only 21.19 tokens/s, greedy MTP n=8 59.46 tokens/s. Filled 261.5K cache: 12.06 tokens/s. Prompt processing at 32K context with a flash attention patch: 815.64 tokens/s (759.38 before the patch).
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B Uncensored at 17.89 t/s generation and 230.07 t/s prompt eval on a two-card rig of RTX 3060 plus RTX 2070 (about 20GB VRAM total).
Setup is Ollama in Docker with Q4_K_M GGUF, q8_0 KV cache, flash attention on, 68115-token context and all 65/65 layers on the GPUs; the host is an i5-13600 with 48GB DDR4.
The prompt eval figure is sustained over a 29,396-token run lasting 27m23s, with 19,489 MiB of VRAM in use.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
long-context
User reports Qwen3.8 27B at 25 tok/s generation with 110K context on a 12GB VRAM, 16GB RAM machine.
Setup is llama.cpp with an IQ3_XXS GGUF quant, q4_0 KV cache, flash attention on, -ngl 62 and 110000 context; prefill runs 750 to 1200 tok/s.
Other context points: 180K at 8 tok/s, 150K at 11 tok/s, 120K at 17 tok/s, 90K at 35 tok/s and 60K at 36 tok/s. The gains came from dropping the display from 4K to 1080p and turning off Moonlight streaming to free VRAM. MTP took too much VRAM to be usable.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User compares six local runtimes on an Apple M5 Pro with 64 GB of unified memory, all running Qwen3.8-27B with a short lighthouse-keeper prompt of about 44 input tokens and 400 output tokens. TensorFold app beta.8 with engine 1.1.0 and an MLX 4-bit model plus DFlash2 drafter was fastest at a mean 36.0 tok/s (36.2 and 35.7 across two runs).
Other means were LM Studio 0.4.26+4 at 33.5 tok/s, MLX-Serve 26.10.2 at 30.1 tok/s, Ollama 0.40.6 at 27.0 tok/s, Magnitude 0.2.9 at 21.0 tok/s and oMLX 0.7.0 at 16.0 tok/s.
User notes the comparison is not apples-to-apples since model packaging, quantization, speculative decoding and runtime defaults differ, and that TensorFold's hardware qualification does not list the M5 Pro.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User compares Qwen3.8 Flash-Next on Strata against a Qwen3.8 27B build for the RTX 5080 on NInfer, reporting 89-112 t/s decode on the 27B.
Setup is NInfer on an RTX 5080 with 32GB DDR5, vision flags removed and thinking budget raised to 4096; the user plugged the display into the iGPU and reports 94.6% VRAM usage while using the desktop.
The user says the 27B solved in one prompt a coding task that the Flash-Next coder and an IQ2_XS build needed multiple prompts for, and calls it insanely fast at prefill and decode.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 Flash-Next at roughly 100 tok/s (90-110 through the server) on an M5 Max 128GB with the new ijah engine.
Setup uses EXL3 trellis weights at 5b-6/8b, about 6% behind oMLX but with 5-bit experts.
User says the quant is about 4-5x closer to bf16 than Jundot's oQ4e (KLD 0.033/0.009/0.015 vs 0.128/0.036/0.078) and about 2x closer than Sushi-4bpw. With Inco's DFlash2 drafter the user reports about 120 tok/s but has not verified it yet.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports a Swift-1.5 Qwen3.8 model at about 95 t/s on dual RTX 4090s after updating Strata from 1.45 to the 2.0 beta.
Setup is Strata with Q3S weights; no context length or KV cache quant stated.
The same setup previously reached around 155 t/s on Strata 1.45, so the user asks whether others see the same slowdown.