Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingcreative-writing
User reports Qwen3.8 Flash-Next at 70 t/s generation and 1.3k t/s prefill on 4x V620 (128GB total VRAM) at 128k+ context.
Setup is a vLLM fork with AutoRound W4A16 quantization and MTP-2 speculative decoding, on an EPYC 7452 with 256GB DDR4. Prose generation runs at 60 t/s.
Power draw is 700-900W during prefill and 500-600W during decode. User was initially disappointed with Qwen3.8 27B speeds but is satisfied with Flash-Next.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B at up to 45 t/s generation and 450 t/s prefill on 2x Tesla P40 with fresh context.
At 150K+ context prefill falls to around 120 t/s and generation to 12-16 t/s.
User notes that running two agents concurrently breaks prefix caching and KV cache sharing, and asks whether a single-supervisor harness with sequential subagent handoffs exists.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next at 31.05 t/s generation and 558.83 t/s prompt processing on an M3 Ultra 512GB.
Setup is llama.cpp through llama-swap with the unsloth Q4_K_XL GGUF at 256,000 context, tested with llama-benchy.
Prompt processing measured at pp1000 @ d1000 with 3312 ms time to first response; generation measured at tg500 @ d1000 with a peak of 31.67 t/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B at 20 t/s generation and 440-460 t/s prompt processing at 163k context.
Setup runs on an RTX A3000M 12GB plus an RTX A2000M 8GB with P2P inside a NAS, using the GSQ-RCO IQ3_S quant.
Prompt processing is given as a range of 460-440 t/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8 27B at around 50 t/s on a CMP 170HX, running at Q8 or BF16.
The user finds the model hallucinates classes and APIs in 90%+ of answers to domain-specific enterprise Java questions, while DeepSeek and Microsoft Copilot produce working code 99% of the time.
The user asks whether Qwen's agentic optimization is responsible and requests tips for a harness, system prompt, or skills to reduce hallucination.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B NVFP4 at 178 t/s on an RTX 5090 after undervolting the GPU and capping CPU power at 50%.
Setup uses the Ninfer inference engine. GPU memory was overclocked by 2400 MHz and the GPU was undervolted; CPU power was limited to 50% with no measurable TPS loss. A systemd service polls GPU usage and applies the CPU cap automatically during inference.
Stock settings gave 172 t/s in synthetic tests; undervolting and VRAM overclocking raised this to 178 t/s, and DFlash2 speculative decoding currently yields 200 t/s. GPU power dropped from 600 W to under 450 W and GPU temperature from 75°C to 62°C.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
multilinguallong-contextvision
User reports Qwen3.8 27B at about 100 t/s on a single RTX 5090, but only about 82k tokens of usable context despite configuring 253,952.
Setup is SGLang serving the Huihui-Abliterated NVFP4 MTP VL build with an fp8_e4m3 KV cache, NEXTN speculative decoding at 3 steps and 4 draft tokens, flashinfer attention, chunked prefill of 1024, and a 16 GB hierarchical cache.
User asks how to tune the setup for more context without losing multimodal capability.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8 Flash-Next at 90-100 t/s decode and 3000 t/s prefill on two RTX 5090s across two machines.
Setup is llama.cpp with RPC over a ConnectX-5 100G link, using Unsloth Q2 quants.
User sees little practical reason to use this over the 27B model and asks what others get with 2x 5090 in one machine or a 6000 Pro.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8 27B GSQ-RCO at 31.7 t/s on a single AMD Radeon RX 9070 with 16 GB VRAM at 96k context, of which 38k were used.
The GSQ-RCO GGUF quant scored 88.84/100 on llm-bench.io, the highest quality score recorded in over 1100 community benchmarks.
The user notes it is not the fastest but sufficient for most coding tasks and praises the quant for preserving quality.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingcreative-writing
User reports Qwen3.8 27B Flash-Next at 28 t/s generation and around 250 t/s prefill on a Strix Halo 128GB at a 131k context limit.
Setup uses the UD_Q2_KD_XL GGUF quant. The model was used for web game development with an MCP for handheld console input standardization and the Blender MCP for model creation.
User compares it against Gemini Flash 3.8 on an identical one-shot backrooms game prompt; Gemini finished in about 5 minutes but produced a low-effort result, while Flash-Next took 2 days including QA and play testing and produced a much more detailed and immersive game, though input was inversed. User says it feels like Opus 4.8 at home and wants a Qwen 35B MoE with engrams at 40 t/s decode and 262k context to go fully local.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B at 5,809 t/s prefill and 276 t/s decode on an AMD Radeon AI PRO R9700 32GB.
Setup is a custom vLLM MXFP4 build with online NVFP4 to MXFP4 conversion, using the Unsloth Dynamic v3 NVFP4 quant.
Results come from BetterBench.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B at 173 to 179 t/s decode on an RTX 5090, with 0.79s first token at about 8K prompt and four concurrent workflows done in 8.15s.
Setup is vLLM with NVFP4 weights, fp8 KV cache, 262,144 context, MTP speculative decoding with 3 tokens, PIECEWISE cudagraphs, and prefix caching.
The same 27B family also runs on a Radeon AI PRO R9700 32GB via Radiance in MXFP4 at 71 to 73 t/s decode, and on the 5090 via ExLlamaV3 at 40 to 70 t/s decode and 600 to 1400 t/s prefill at full 262K depth.
User runs SWEBench locally with llama.cpp and an Unsloth Qwen3.8 27B IQ4_NL GGUF on an RTX 3090, at 229,376 context with a q8_0 KV cache.
Setup uses --jinja, -ngl 999, flash attention, mmap load mode, and a bf16 mmproj; speculative decoding is disabled.
The 500-test run took 8 days and produced 9 LimitExceeded and 103 TimeoutExpired errors, leaving 388 completed tests with 339 resolved and 48 unresolved, an 87.5969% resolve rate. User asks how to reduce errors, improve success rate, and speed up the run without sacrificing context.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8 27B at 3.20 t/s generation and 32.45 t/s prompt processing on a Ryzen 9 7940HS with Radeon 780M iGPU and 24GB RAM.
Setup is Ollama on two machines: a Minisforum UM790 Pro and a home build with the same Ryzen 9 7940HS, both with 24GB RAM. The Minisforum run reached 4.73 t/s generation and 20.52 t/s prompt processing.
Both runs stopped mid-task and follow-up prompts returned no output. User expected low throughput but not the failures.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codinglong-context
User reports Qwen3.8 Flash-Next at 70.0 t/s generation and 1,386 t/s prompt processing on 4 Radeon Pro V620 GPUs at 32K context on the coding suite.
Setup is vLLM with the Intel/Qwen3.8-Flash-Next-W4A16-AutoRound quant, using a custom RDNA vLLM fork.
At 64K context the coding suite reached 72.5 t/s generation and 1,369 t/s prompt processing; at 128K it reached 68.3 t/s and 1,296 t/s. The regular suite measured 57.9, 59.5, and 56.1 t/s generation at 32K, 64K, and 128K respectively.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
tool-uselong-contextagenticcreative-writing
User reports Qwen3.8-Flash-Next on one RTX PRO 6000 Blackwell 96GB, with the official SGLang NVFP4 image cutting time to first token at roughly 254K context from 34.9s to 22.4s and raising prefill from 7,284 to 11,352 tok/s.
Setup is the lmsysorg/sglang:dev-qwen38-next-local image with the RadixArk NVFP4 checkpoint, a 262,144-token window and one slot; the previous build streamed the 47.68 GiB per-layer embedding table from NVMe while the official image pins it in host RAM.
A separate cache test dropped first-token time from 21.7s cold to 0.44s cached. Long-context recall scored 81/81, BFCL single-turn tool accuracy 85.2% versus 49.5% multi-turn, and tau2-bench telecom completion 68.1% with 41.2% passing all three attempts.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next at around 100 t/s prefill and 18-20 t/s decode on 2x RTX 5080 with 128 GB DDR5.
Setup is llama.cpp with the unsloth UD-Q5_K_XL GGUF, a Q8_0 KV cache, 262,144 context, MTP speculative decoding with a Q4_K_M draft model, and ncmoe42 offloading 42 MoE layers to CPU.
User asks whether switching to vLLM or SGLang would give better performance, noting the scarcity of NVFP4/FP8 GGUF files.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8 27B at 100 t/s on a 4x RTX 3090 machine with 96GB VRAM and 384GB DDR4 RAM.
Setup uses DFlash2 speculative decoding with BF16 weights.
User also tested Qwen3.8 Flash-Next at 4-bit, getting about 70 t/s, and found it produced visual glitches and HTML issues compared to 27B. Tried Intel AutoRound W4A16, generic W4A16, and Unsloth UD-Q4_K_XL quants for Flash-Next.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingcreative-writinglong-contextvision
User reports Qwen3.8 27B Huihui abliterated NVFP4 at 205 t/s decode on a single RTX 5090 32GB, with prefill around 9,000 t/s at 8K context.
Setup is NInfer on Windows 11 + WSL2 with fp8 KV cache at 196,608 context, MTP speculative decoding with 3 draft tokens and --lm-head-draft, vision enabled, weights about 19.7 GB.
Decode varies with MTP acceptance: 253 t/s on predictable text, 205 t/s on coding, about 120 t/s on creative prose, and 60-80 t/s with MTP off. At 128K context decode is about 115 t/s and prefill about 4,500 t/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B NVFP4 at 120 tok/s at concurrency 1 on a single AMD Radeon R9700, with prefill at 3,619 tok/s at 16000 depth.
Setup is a vLLM fork (radiance-vllm-mxfp4) with Unsloth's Qwen3.8 27b NVFP4 weights on one card.
Median decode varies by category from 67.1 tok/s on chat to 153.1 tok/s on json, while update p50 stays between 34.2 and 42.5 ms. Concurrency scales to 471 tok/s at 8 concurrent requests.