Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B Bonsai 2 at PQ2_0 (2.13 bpw, 7.21 GB) on one RTX 3090 with the llama-cpp-prism ternary fork, prefilling at 834.8 t/s at 10K.
Setup holds the full 262144 context on the single card at 17.1 GB with q8_0 KV; the quant drops the MTP head, so the run is un-drafted. Decode is given per workload only, 42.76 t/s narrative and 41.01 code, with 141 ms TTFT.
The same harness PR adds an exl3/TabbyAPI engine on the 2x 3090 reference rig, whose CPU-computed MoE slugs decode at 50.5 and 39.3 T/s at 204800 context with Q4 KV.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B EXL3 3.00bpw on a bare RTX 3090 24 GB through SGLang with the sglang-exl3 plugin, with cold prefill at 904 tok/s.
Setup uses fp8 KV cache, MTP speculative decoding on, thinking off, one request, at the full 262,144-token window; the 2.4 GB token embedding sits in pinned host memory.
Decode is given per workload: 98.8 t/s on prose, 143.3 on code and 89.6 on prose with a 32k prompt. A Qwen3.6-35B-A3B EXL3 recipe on the same card reached 251.9 prose and 352.1 code. A prior AWQ-INT4 27B recipe with no draft is cited at 45.1 t/s prose.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next at 38-40 t/s on three RTX 3060 12GB cards in a mining rig frame with Strata, against 13.2 t/s with llama.cpp.
Setup is an IQ3 quant through Strata, with 64GB DDR4 and an 8th gen i7; at IQ3 the model does not fit in 36 GB of VRAM, so part of it runs from system RAM.
The user asks whether others run dated mining hardware with similar success.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagenticlong-context
User reports Qwen3.8-Flash-Next on an M5 Ultra 96GB (64-core GPU) prefilling a cold 3.7k-token prompt at 1,354 tok/s.
Setup is mlx-serve 26.10.1 with iQ-MLX 3.3 bpw, 131,072 context, native MTP adaptive decoding, PLD and a hot prefix cache, one request at a time.
Decode is given per workload only: 227.5 tok/s on copy, 172.8 on new code and 106.4 on novel prose, with speculation on. Cold prefill rose to 4,083 tok/s at 25.2k. An agent replay ran at 125.8 tok/s text-round decode.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codinglong-context
User benchmarks Qwen3.8 27B at ~31.3 t/s generation and ~997 t/s prompt processing on a single modded RTX 3080 20GB.
Setup is llama.cpp with Q4_K_XL GGUF and Q8_0 KV cache at ~98k context. Native MTP raises generation to ~56.9-60.7 t/s but cuts context to ~68k.
Adding an RTX 3070 8GB via layer split reaches ~223k context but drops generation to ~27.8 t/s; tensor parallelism over the chipset x4 slot hits ~59.7 t/s generation with a ~20% prompt-processing penalty, while moving the second card to a CPU-attached Oculink x4 link recovers ~976-1,005 t/s prompt processing at ~60.4 t/s generation and ~204k context.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codinglong-contexttool-use
User reports Qwen3.8-27B at 125–134 tok/s decode (greedy) on a single AMD Radeon AI PRO R9700 32GB, running on Windows 11 + WSL2 with a Ryzen 9 9950X.
Setup is vLLM with AMD's official MXFP4 checkpoint (amd/Qwen3.8-27B-Quark-AWQ-MXFP4), DFlash2 speculative decoding with a FP8 drafter proposing 7 tokens per step (~60% acceptance, ~5.3 tokens per forward pass), and RDNA4 kernels for FP8 paged attention and gated-delta-net layers. Default context window is ~216k tokens, extendable to ~260k with -Long.
Prefill is ~2,750–2,970 tok/s with 0.7 s time to first token on a 1.9k-token prompt. Decode deep in context is 165 tok/s at 32k, 136 at 98k, and 115 at 258k. Long-context recall retrieved 8/8 planted facts from a 258k-token prompt. A comparison llama.cpp setup (IQ4_XS GGUF + speculative decoding) does about 52–68 tok/s on the same card, though measured with a different prompt. Spilling into shared memory drops performance to ~10 tok/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagentic
User reports Qwen3.8 Flash-Next at 25-30 t/s at 130k context on a GPD Win 5 with 64GB Strix Halo, docked.
Setup is a llama.cpp fork with IQ3-XXS quant and Q4_0 KV cache, using MTP speculative decoding with a 64k vocab limit and Q5 hyper-connections.
On battery the same model runs at 18-24 t/s with 70-80 minutes per battery pack. Qwen3.8 27B runs at 12-19 t/s at 8k context, and Tiel 35B-A3B-Q6_K_XL averages around 50 t/s at 64k context. A desktop GMKtec EVO-X2 with the same engine and model gives 30-35 t/s at that depth.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports 1-2 tokens per second on coding tasks with Qwen3.8 27B on an RTX 3060 12GB.
Setup is Ollama on Windows 11 with WSL, with 72GB system RAM and a Core i9 14900k. The user also mentions Qwen3-Coder 30B, Gemma 4 31B, and Qwen2.5-Coder 7B.
The user says every coding task is painfully slow, with 15-minute thinking times, no output, and stream timeout errors. They ask for help setting up a usable local coding workflow.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 Flash-Next decoding at about 45 t/s on a 128GB Strix Halo.
Setup is Halogen 0.17.2; the user says decode stays consistent at that rate even at high context.
The user calls it crazy good for a 177ish billion model and praises the engine's rapid development.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports running Qwen3.8-Flash-Next UD-Q4_K_XL at 262K context on a Strix Halo 128GB (Ryzen AI Max+ 395) with a 96GB GPU carve-out, using Gufo (a Windows-port fork) with Unsloth Studio as client.
Setup uses a 64GB minimum / 192GB maximum page file and a source patch to Gufo's snapshot budget (set to 16GB via environment variable) to fix a Windows memory misread that capped the cache at ~264 MiB. Disk cache is off.
Performance is about 900 to 1,000 tok/s prompt processing and 35 to 55 tok/s decode depending on context length. First token at 120K+ tokens now arrives in 2 to 5 s instead of 40 s or more. User is happy with the setup.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8 27B at IQS3_XXS running at 6 tok/s on a Thinkpad X9 15p with an X7 358H and 32GB RAM.
No engine or context length is stated.
User finds the speed slow and asks for a faster 20-30B MoE or dense alternative for refactors and menial work on personal projects.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingvisionmath
User reports Qwen3.8-27B at 47-48 tok/s on an M1 Ultra 64GB.
Setup is oMLX 0.7.0 with oQ4e quant and MTP fixed depth 2.
MTP off gives 27.4 tok/s, adaptive depth 44.3 tok/s, and DFlash2 23.6 tok/s. The model is also used for vision, scoring 35/35 on a 35-item suite.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
summarizationagenticlong-context
User reports Qwen3.8 27B at 23 t/s generation and 304.2 t/s prompt processing on an M2 Ultra 128GB Mac Studio.
Setup is oMLX 0.6.3 with oQ4e 4-bit quantization at workload depth 65536, with 16.7GiB of weights and peak memory rising from 25.3 to 36.2GB.
The 304.2 PP is the ANE-on arm; the separate GPU-only baseline is 242.2 PP and 23.8 TG, and the source calls the decode change within variance. Depth 65536 is not a verified configured maximum context.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8 27B Optimized Speed at 58.7 tok/s on a MacBook Pro M5 Max.
Setup is MTPLX with a 4-bit dynamic MLX quant and native multi-token-prediction speculative decoding at depth 3, single stream, 262,144-token context window, 20.4 GB download and 23.6 GB peak unified memory.
The same coding task ran 55.5 tok/s inside the MTPLX Mac app, and long reasoning at xhigh gave 35.1 and 37.3 tok/s on 28k and 20k token answers. For comparison on the same task, Qwen 3.6 27B Optimized Speed V2 ran 59.9 to 60.1 tok/s and oMLX 0.5.7 with its own Qwen 3.8 4-bit MTP quant ran 63.3 tok/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports a Strata fork (Strata Void) with task-parallel requests running Swift 1.5 Qwen3.8 Flash-Next at around 40-45 tok/s single-stream on three Intel Arc GPUs (B70 32GB, B65 32GB, B60 24GB, 88GB total VRAM).
Setup is Strata with IQ4_XS weights and a 32K context window; all expert weights are in VRAM with no RAM spilling.
The fork splits complex requests into parallel subtasks and synthesizes the results: 10 complex tasks took a 181s median as a normal request, 68s median with four parallel subtasks plus synthesis, and 78s median in AUTO mode. The user notes this does not raise single-stream token generation speed and uses more total compute.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingagenticlong-contextsummarization
User benchmarks Qwen3.8-Flash-Next on a DGX Spark (GB10, 121 GB unified memory) against Claude Opus 5.5 across 21 graded coding tasks. Local model matches Opus on 11 everyday tasks (11/11 each) but is roughly 5-8x slower, and scores 7/10 on hard tasks versus Opus's 10/10.
Setup is vLLM 0.30 with NVFP4 weights and MTP speculative decoding, about 30 tok/s on long generations. Harnesses tested were a minimal 3-tool loop, Claude Code 2.1.285, and Qwen Code 0.24.7.
User notes the failure mode was over-verification rather than inability, and that harness choice moved the hard-tier score as much as settings did. A real-world test built a 150-line spec wedding microsite in 88 minutes, about 5.4k lines across 30 files.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B Uncensored-HauHauCS at about 130 t/s decode on an RTX 4090.
Setup is NInfer with INT8 quantization, 262K context, and a model size of about 17 GB.
The user notes this is a port in progress, with a CPU build possibly running overnight; the 130 t/s is an expected figure rather than a measured result.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8-Flash-Next at about 40 t/s on unspecified hardware.
Setup is Strata with a GSQ-RCO-IQ1_M quant.
The user is asking which VSCodium extension to use for AI coding and does not name the GPU or context length.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagentic
User reports running Qwen3.8-Flash-Next-UD-IQ3_XXS at Q8_0 KV-cache with llama.cpp, getting 30-40 t/s generation and 530 t/s prefill at 220,000 context.
Setup is llama.cpp with UD-IQ3_XXS quant and Q8_0 KV cache, limited by VRAM.
User stopped using Strata after 12 hours of coding due to reliability concerns, citing hallucinations and coding errors, but notes Strata greatly increased prefill speed and is a worthwhile project.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingvisionlong-context
User reports Qwen3.8-Flash-Next at ~60 tok/s serial decode and ~730 tok/s prefill on an M4 Max 128GB.
Setup is mlx-serve 26.8.11 with a mixed 4-bit/8-bit MLX pack, ~75 GB resident, prefix cache on, images included. With MTP enabled decode reaches 78 tok/s (+41% on code, a few percent slower on prose). A needle at 24.8k tokens was recovered with sparse attention engaged.
The pack includes the MTP head and vision tower; the 51B n-gram table ships as a 32 GB mmapped file that is not resident. The author recommends a re-quantized iQ-MLX-4.7bpw version instead.