Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
visionlong-contextcoding
User reports Qwen3.8-27B at 129.0 t/s decode and 786 t/s prefill on an RTX 3090.
Setup is sglang-exl3 v0.7.0-ampere with exl3-3bpw weights, MTP speculation, vision, and a 208k context window.
Protocol acceptance records prose 96.2 and code 141.1 tok/s per stream, cold prefill 1,523 tok/s at 32k and 710 at 209k, and a 4096² image TTFT of 22.5 s. A companion Qwen3.6-35B-A3B recipe at 256k reached 309.6 t/s decode and 1,907 t/s prefill.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
agenticcodinglong-context
User reports Qwen3.8-Flash-Next Q2_0 at 44.0 tok/s decode and 328 tok/s prefill at 4K context on a 2021 M1 Max with 64 GB unified memory.
Setup is antirez's ds4 engine (forked for Apple Silicon) with Q2_0 GGUF weights, MTP speculative decoding on, single request, model fully in unified memory. Decode holds 38.8 tok/s at 256K and 35.4 tok/s at 398K; prefill falls from 328 to 292 tok/s over the same range. IQ3_XXS (44 GiB) gives 35.2 tok/s at 4K and 31.7 at 259K.
Sustained five-prompt loop for 5 minutes: 43.2 tok/s first quarter, 43.1 last, 73.1 C, 0.70 J per token. In OpenCode sessions (162 requests, context up to 371K) median decode is 44.7 tok/s under 64K and 38.1 at 128-192K. User compares against a Splash port of Qwen3.8-27B (14 tok/s at 128K+ vs 38) and a DGX Spark running vLLM NVFP4 + MTP (36.8 tok/s prose, 45.8 code). DeepSeek V4 Flash (81 GiB Q2, experts streamed from SSD) runs at 11.8 tok/s at 4K on the same Mac.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingagenticvision
User reports Qwen3.8 27B at around 80 t/s on an RTX 5090, running Q6 with 160k context, medium thinking and vision enabled.
Setup uses the Hermes agent harness with Searxng in Docker for web browsing.
User finds it slow for real-world coding tasks, citing 15 minutes for a website one-shot versus 3-5 minutes with Claude, and questions whether the model's overthinking makes it weak for coding.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User compares two local Qwen models against Claude Opus 4.6 on three Python coding tasks, scoring hidden tests and code review. The local Qwen3.8-Flash-Next Coder model tied Opus 4.6 at 92.7 out of 100, while Qwen 3.8 27B Q6 scored 87.0.
Setup is two RTX 5060 Ti 16GB cards (32 GB VRAM total) on an Intel Core i5-14400 with 48 GB DDR4 RAM, Windows 11. The Coder model is a pruned Strata variant keeping half the experts per layer, as an IQ1_M GGUF, running between 50 and 90 tokens/s, around 60-65 on average. The 27B Q6 Unsloth quant runs at a steady 50 tokens/s.
User notes one run per model, differences of 2-3 points are meaningless, and the Coder model was graded harder with more weird-input cases. The review was done by Claude Fable 5.1 with public grading sheets.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 Flash-Next 176B at 11.09 tok/s decode on an RTX 3080 Laptop 16GB with 32GB system RAM and SSD.
Setup is TensorSharp with MoE-aware unified scheduling across cache, VRAM, system RAM, and SSD; quant and context length not stated.
User compares against Strata, which reached 10.24 tok/s decode and 62.15s whole-process time versus TensorSharp's 16.54s; TensorSharp GPU peak was 14,832.5 MiB and OS peak working set 19.74 GiB.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B at 47.2 t/s decode on one Tesla V100-SXM2-32GB (250 W) in an X99 box with 2x Xeon E5-2690 v4 and 251 GB RAM.
Setup is llama.cpp (V100 tensor-parallel fork) with IQ3_S GGUF weights, q8_0 KV cache, 128k context, and MTP self-speculation with up to 3 draft tokens per step.
A 64k context run with f16 KV reached 52.5 t/s with 91.9% draft acceptance; the 128k run had 88.0% acceptance. Per 8k-token stretches the rate ranged 46-66 t/s at 64k and 41-57 t/s at 128k.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B-Uncensored at 833.37 prompt tok/s and 43.34 decode tok/s on two NVIDIA Tesla V100 GPUs.
Setup is llama.cpp with Q8_0 weights, F16 KV cache, 262,144-token context, tensor split 1,1, and the built-in MTP head with draft length 3.
The matching DFlash2 Q8 candidate measured 607.82 prompt tok/s and 33.00 decode tok/s, so MTP remains the production default. A single-V100 Q4_K_M profile with Q4_0 KV and MTP draft length 1 reached 44.201 tok/s on a 512-token short test and 13.001 tok/s on a 255,000-token capacity run.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagentic
User reports Qwen3.8-27B at 47.0 t/s generation and 854 t/s prompt processing on an Apple M5 Max with 128 GB unified memory.
Setup is mlx-vlm with the 4-bit MLX quantisation and MTP speculative decoding enabled, 15.5 GB resident. The 8-bit quant reaches 28.7 t/s generation and 883 t/s prompt processing at 28.1 GB resident.
MTP speculative decoding is reported as worth 2.2x (15.5 t/s to 34.1 t/s at 8-bit). Decode falls to 36.1 t/s at 8K context, and time to first token climbs to about 14 s at that depth.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B at 135.4 tok/s decode on a bare RTX 4090 24 GB.
Setup is SGLang with EXL3 3bpw weights, 207,356-token KV pool, 22.9 GB peak, 239 ms TTFT over 3 samples.
A separate depth sweep on the same card gives 184 tok/s code and 126 tok/s prose at ~50 prompt tokens, falling to 115 and 91 tok/s at 133k-190k tokens; cold prefill was 2,112 tok/s at 19k and 1,636 at 134k. The recipe is the unchanged RTX 3090 SGLang EXL3 config, and it now displaces a TabbyAPI SC4bpw recipe that measured 86 tok/s. No coherence ladder, soak test or 3 bpw vs 4 bpw quality comparison was done.
Ryzen AI Max+ 395 96GB · GSQHalo.cpp · 131,072 ctx
generation:
44.2 tokens/s
prompt processing (prefill):
1386.0 tokens/s
quant:
IQ3_XXS (GGUF)
kv:
f16
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next at 1386 t/s prefill and 44.2 t/s decode on a 96GB GMKtec EVO-X2 (Ryzen AI Max+ 395).
Setup is a fork of strix-llama.cpp called GSQHalo.cpp with GSQ-RCO IQ3_XXS GGUF quants, f16 KV cache, 128K context, MTP on, and KV cache stored on SSD.
Without MTP the same run gives 1446-1457 t/s prefill and 24.3-24.5 t/s decode. Two 256K slots with MTP give 1067 t/s prefill on a 250K prompt and 14.1 + 16.6 t/s decode on both slots at once.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8-Flash-Next at 110.6 t/s generation and 2773 t/s prefill on two RTX 5080s with a Ryzen 9 9900X, using a custom Strata fork.
Setup is Strata with UD-Q4_K_XL, 64K context, INT8 K/V, 8K prompt chunk, MTP on and expert adaptation on. Upstream v0.1.38 reached 78.1 t/s generation and 2016 t/s prefill with layer split, and 70.3 t/s generation and 1522 t/s prefill with the host peer tier.
With static caches custom was about 26% ahead of upstream layer split (55 vs 44 t/s). Letting the second GPU compute experts during prompt processing doubled custom's 32K prefill (1366 to 2731 t/s), and a 16K chunk brought it to 3724 t/s. Putting the expert tier on the x8 PCIe card improved 4K prefill by 58% and 32K by 34%. The saved answer came from the custom fork, so it may favor its drafts.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8-Flash-Next (Swift 1.5 IQ2_XS) at 143 tok/s code generation on an RTX 5080 + RTX 4060 Ti with 32 GB system RAM.
Setup is a dual-GPU fork of the Strata engine at 256K context, with experts that don't fit in VRAM read from the SSD and a fine-tuned draft layer for speculative decoding.
The same machine with the 5080 alone reached 29 tok/s code and 333 tok/s on a 32K prompt; the fork with both cards reached 1,940 tok/s on the 32K prompt. In real use with the Pi coding agent it peaks above 200 tok/s (209 so far) and a 150K-token context reads at about 1,850 tok/s. Perplexity was 7.24 vs upstream's 7.28.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingagentic
User reports Qwen3.8-27B at 40 t/s on an RTX 3060 12GB.
Setup is Beellama with an IQ3 head and KvarnN KV quantization at 3/3, 80k context, and Dflash2 speculative decoding.
The model is a custom hybrid combining Escha Labs W2 body with a Lowgpu IQ3 head, 8.65GB total size. User also tested on a 4070 Ti and 50-series cards.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next at 69 t/s at 98k tokens on an RTX 4070 12GB and Tesla V100 SXM2 16GB.
Setup uses IQ2_XS quant with 32GB DDR4 RAM and a 3500MB/s SSD, with CPU offload.
Speeds scale from 37 t/s fresh to 69 t/s at 98k tokens. User also gets 45-50 t/s on a 27B Q4_K_M model and asks whether IQ3_XXS would fit.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User asks whether Qwen3.8 Flash-Next can run on an RX 9060 XT 16GB with 32GB DDR5-6000.
They currently run Swift-1.5-Qwen3.8 at IQ3_S at 30 t/s on the same machine.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen 3.8 Flash Next at 10 t/s decode and almost 100 t/s prefill on an RTX 2060 Laptop 6GB with 32 GB RAM.
Setup is Strata with q2_0 quant and Q4_0 KV cache at 50k context, using draft-vocab en and 100 MiB VRAM reserve.
User notes prefill is the bottleneck for large prompts, and that Qwen A35B A3B has around 5x faster prefill and allows 100K context without KV quantization.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8 Next at 45-50 t/s on an RX 7900 XTX with slow DDR4 system memory, using Strata with IQ3_XXS weights just under 80GB.
Setup is Strata with IQ3_XXS weights, MTP speculation enabled, and CPU offload to system RAM. Llama.cpp with tuning maxed out around 22.5 t/s on the same rig.
User notes similar results for 12GB and 16GB cards, faster numbers for DDR5 owners, and recommends against the Q2 weights.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
agentic
User reports Qwen3.8 Flash-Next at 40 t/s average on agentic tasks on Strix Halo 128GB.
Setup is Gufo with UD_Q4_XL quant, VRAM set to 96GB.
User notes the speed holds at higher context lengths and recommends setting VRAM to 96GB on Windows.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User compares Qwen3.8 27B across four setups on an RTX 5090 at 120k context, reporting decode speeds of ~147 t/s (ninfer non-NVFP4 with MTP), ~141 t/s (llama.cpp Q4_K_M with MTP), ~155 t/s (ninfer NVFP4 with MTP), and ~68 t/s (llama.cpp Q4_K_M without MTP).
Setup is llama.cpp with Q4_K_M GGUF, thinking on at xhigh, default sampling, one run per setup.
Stock llama.cpp without MTP is less than half as fast as ninfer, but MTP lifts it from 68 to 141 t/s. NVFP4 had the highest t/s but wrote the most tokens, finishing third on wall-clock time. All four runs produced working games; only the NVFP4 build could be won legally. User is sticking with the non-NVFP4 ninfer build.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User compares local model choices by memory budget, recommending Qwen3.8-27B at Q4 for 32 GB or more of VRAM and Qwen3.8-Flash-Next on Strata otherwise.
On 2x RTX 5070 Ti 16 GB with 96 GB RAM, the user measures Qwen3.8-27B under ninfer-tp2 with MTP/DFlash at roughly 180-290 t/s.
Other reported figures include a single 5090 at about 150-210 t/s with MTP3 and 260-290 t/s on code with DFlash2, a single R9700 at about 50-65 t/s under llama.cpp or vLLM with MTP, and 2x R9700 at about 185 t/s under vLLM MXFP4 with speculative decoding. For Qwen3.8-Flash-Next on Strata the user measures 62-72 t/s decode on one GPU and 80-90 t/s on two, with prefill around 2,400 and 3,000 t/s at 16k.