Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports running Qwen 3.8 Flash-Next IQ3_S and Swift 1.5 IQ3_XXS on an RX 9060 XT 16GB with a 7800X3D and 64GB RAM.
Setup is Strata, with Flash-Next at a 130K context window and Q8 KV cache, and Swift 1.5 at 262K context and Q4 KV cache.
Token generation stayed between 40-55 tk/s across the whole context window for both models. Prompt processing was around 800 tk/s on resumed sessions and around 900 tk/s during compaction. Neither model could build the requested Figma plugin in one shot, but the user is happy with performance and plans to stay on the 9060 XT.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B (Swift-1.5 MXFP4 GPTQ) at 154.4 tok/s weighted single-stream decode and 3,927 tok/s prefill on one AMD Radeon AI PRO R9700 32GB.
Setup is vllm-sly-radiance with MXFP4 weights, FP8 KV cache, 65,536 context, DFlash2 speculative decoding with a W4A16 drafter (k=7), 4,096 prefill chunk, prefix caching off.
Prefill falls to 3,370 tok/s at 47k tokens; aggregate throughput is 480 tok/s at 8 concurrent requests and 512 tok/s at 16. Per-category decode ranges from 90.5 tok/s (prose) to 210.8 tok/s (json).
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B running on the M6 Neural Engine via ANEMLL Forge 0.2.1, with about 27 tok/s decode and 123 tok/s prefill at 96K context on a 32GB M6 Mac Mini.
Setup uses an INT8 K/V cache and a smaller drafter sharing the target's output head, with about 21.4 GB total memory at 96K context.
Decode varies by content: JSON 35–62 tok/s, code 18–32, prose 10–17. Prefill is 260 tok/s at short context and 62 at 192K, where a smaller prefill batch of 32 rows is used. The user calls 96K the sweet spot for size and speed on 32GB.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
coding
User reports running a pruned Qwen3.8-Flash-Next Coder GGUF at 70 t/s prompt processing and roughly 8-9 t/s generation on an 8 GB GPU with 24 GB system RAM.
Setup is llama.cpp with a 3.5 BPW GSQ RCO quant, 262K context, most MoE experts kept in RAM and the n-gram table on disk, with selective MoE offloading by regex rather than --n-cpu-moe.
The generation figure is given as a range (8~ T/s) and is limited by PCIe 3.0; the user notes a first run is slower due to MMAP and a second run is needed for accurate numbers. Half the experts are removed, giving an effective 1.89 bits per parameter over the original model.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next at 178 tok/s decode and 4,828 tok/s prefill on a 64k prompt on two RTX 5070 Ti 16 GB cards.
Setup is the user's own capy-flash engine (a fork of NInfer) with NVFP4 weights, tensor parallel across both cards and a shared VRAM expert cache whose misses run on the CPU, on a Ryzen 9 9900X with 96 GB DDR5.
Figures are one request at a time, decode measured with LocalMaxxing reasoning-v1. The engine also runs Qwen3.8-27B.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
long-contextsummarization
User reports Raon-1 mini preview at 96 tok/s on an RTX 3090 while running a Korean RAG benchmark over 300 questions and 64 PDFs.
Setup details are not stated; the run used the same top 5 retrieved pages and prompt for every model, with thinking on and temp 0.
Raon-1 scored around 74-75% and looped on 16 of 320 questions at temp 0 with thinking on. Qwen3.8 27B scored 77.3% versus Opus 5.5 at 76.6% (p=0.82 on McNemar), and thinking off dropped Qwen about 6 points. gpt-oss 20B went from 57.5% to 69.6% after raising the token limit from 1024 to 8192. BM25 hit@5 was 56% with whitespace splitting and 80% with a Korean morphological tokenizer. The user notes 11 verdicts flipped across hardware, so a 1 point gap means nothing.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingagenticlong-context
User reports running Qwen3.8 27B Q6 with 210k context on two modded 20GB RTX 3080s, generating at ~45 t/s with prefill at ~500 t/s.
The 40GB of VRAM was filled by the model and context; the build cost $2250 plus $250 for an extra 64GB of RAM, and has run for 6 weeks.
Both throughput figures are approximate single readings. The user's goal is a local coding agent with long full-precision contexts and several agents at once.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next at 236 tok/s single-stream aggregate on four R9700 32 GB cards at TP4, with 1,192 tok/s at 16 concurrent streams.
Setup is radiance 1.11.0 with the fp8-iq4r-moe container (113.6 GiB, 4-bit experts, int8 trunk, bf16 protected experts), fp8 KV cache, MTP head depth 3, 262144 max model length, all weights in VRAM and the 47.7 GiB n-gram table pinned in host RAM.
Decode step is 11.27 ms against a 11.2 ms reference; prefill measured 8.4K t/s at 4.4K and 70K prompt tokens, and single stream with thinking on reached 197 tok/s over 4,096 generated tokens. Concurrency rows are thinking off, 512 tokens each, wall-clock over the whole batch; a 2x TP2 attempt lost.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagenticlong-context
User reports Qwen3.8-27B at 51 t/s average decode and 838 t/s prompt processing across about 1,030 real agentic coding requests on 2x Arc Pro B70 32GB.
Setup is llama.cpp SYCL with tensor parallel across both cards, Q8_0 weights, a DFlash2 drafter at Q4_K_M, 200K context and temp 1.0 on Windows 11.
Decode ranged from 47 t/s at 40-80K context to 61 t/s at 160K+, rising at high context because more drafts get accepted; a fresh 94K prompt prefills at about 1,070 t/s. The user also reports a oneAPI 2026.1 runtime leak that pushed private memory to 92 GB and slowed decode from 64 to 200 ms per cycle, fixed with one SYCL context per GPU.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User compares their own engine, kapitsa, against Ollama 0.40 running Qwen3.8-27B at 4-bit on four Apple Silicon Macs, reporting 140 vs 68 t/s decode on an M5 Max.
Kapitsa uses kflash speculative decoding, where a drafter guesses 7 tokens ahead and the 27B verifies a tree of guesses in one pass; Ollama runs the model with MTP or without speculation.
Other decode results: M5 Pro 81 vs 40, M4 Pro 63 vs 32, M3 Pro 40 vs 23. The user notes Ollama reads prompts faster on the M5 Max, so kapitsa's advantage is in decode. The 27B needs 32GB of memory; a 9B model is offered for 16GB machines.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next at 52.7 t/s generation with multi-token prediction and about 1,250 t/s prompt processing on a 6K-token prompt, on a Strix Halo 128GB box running Windows.
Setup is Rulith Inference 0.6.3 with turboderp's EXL3 4.05 bpw quant repacked as GGUF, q8_0 KV cache, 64 GB VRAM setting with overflow experts in system RAM, 512K context and up to 8 parallel chats. Generation is 28.8 t/s without MTP.
The EXL3 build is 104 GB against 111 GB for UD-Q4_K_XL, with mean KLD 0.022 versus 0.024 and code KLD 0.0044 versus 0.0086; chat and multilingual KLD are a tie. The 39 GB n-gram embedding table is read from disk by rows and never goes to the GPU. Prompt processing is a few percent slower than the UD-IQ4_XS build.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
agenticlong-context
User reports Qwen3.8-27B at 147 tok/s decode on one RTX 5090 under Windows 11, using their own VoidInfer engine.
Setup is EXL3 6 bpw weights with OSCAR INT4 KV cache and VeriCache verification against FP16 KV history kept in host RAM, giving roughly 400k shared context on the single card.
Against NInfer (NVFP4 + FP8 KV) on the same card, VoidInfer is about 25% slower decode (147 vs 190 tok/s) and much slower prefill (3.5K vs 9.3K tok/s at 3K prompt), but claims KL to BF16 of 0.014 vs 0.080 and 99.3% vs 96.0% agreement with BF16.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
agentictool-use
User compares Qwen3.8-Flash-Next NVFP4/GPTQ against Unsloth UD-Q6_K_XL on an RTX 5090 32 GB, with NVFP4 at 166.3 t/s generation and 3,033.4 t/s prompt processing at 4,096 tokens.
Setup is Strata with int8 KV cache and a 131,072-token context ceiling; the speed figures are medians of three 256-token generations, and Q6's speed campaign had its control vector enabled.
On a fresh 84-question MMLU-Pro subset NVFP4 scored 71/84 against Q6's 67/84 in 10m 57s versus 22m 18s, a difference the user calls not statistically established (McNemar p 0.219). Q6 reached 65.6 t/s at 4K and 2,094.4 t/s prefill, and an earlier mixed test had Q6 ahead on IFEval 9/10 to 7/10.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
long-context
User compares Strata against oMLX running Qwen3.8-Flash-Next on a 128GB M5 Max, with Strata decoding at 74 tok/s versus 57.5 tok/s for oMLX at 245K context.
Strata is a one-month-old engine built for this one model, running on Metal.
The comparison holds one request at a time only, which the user notes as the current limitation.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B running at 10 tokens per second on an RTX 4080 Super with an i9-14900K and 64GB of RAM.
Setup is LM Studio serving over the local network, with the model downloaded through the app; quant, context length and KV cache are not stated.
The user is asking what speed to expect as a baseline and how to run the model more effectively, and also wants image input support.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen 3.8 27B at about 4 tokens/s on an RTX 5070 in a laptop with a Core Ultra 9 275HX and 32GB DDR5.
Setup is llama.cpp with Q4_K_M, launched with -ngl auto; anything else crashes.
The run uses 100% of system RAM, about 25% of the iGPU and 0-3% of the RTX 5070, so the model is not running on the discrete GPU.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next at 160 tok/s single-stream on two Radeon AI PRO R9700 cards.
Setup details, engine, quant and KV cache are not stated in the post.
At 8 concurrent users the user reports 470-630 tok/s aggregate. The user made a quant of the model and posted benchmarks and dev notes, and is looking to compare notes with other R9700 users.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingagentic
User reports Qwen3.8 27B at 144 tok/s on an RTX 5090 32GB, with 96GB of system RAM in the machine.
Setup is Ollama with a 260k context window and KV cache enabled; the user does not state the quant.
The user asks whether this is the ceiling, since agentic coding sessions run 8 hours at a time with the GPU at 100% utilization and temperatures around 60 C.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
tool-useagentic
User reports Qwen 3.8 27B at 36.5 t/s on an RTX 4070 Ti Super 16 GB with 64 GB system RAM.
Setup is Ollama on Windows with UD-Q3_K_XL and an 8-bit KV cache at 32K context.
The user says the 8-bit cache alone added about 26% generation speed and Q3+Q8 was about 78% faster than IQ4+Q8; other rows were Q4_K_M at 11.3 t/s, IQ4_XS at 16.3 t/s and IQ4_XS with 8-bit cache at 20.5 t/s. These are short tests on two warmed deterministic prompts, not an accuracy benchmark, and harder-task quality is untested.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User compares Swift-1.5-Qwen3.8-27B at around 13 tokens/sec against Strata on an RTX 4070 12 GB plus RTX 3060 12 GB with 32GB DDR4, running the same instruction runsheet.
Qwen3.8-27B runs Q4_K_M; Strata runs IQ3_XXS with experts streamed from an experts.bin file on an M.2 SSD, context varied from 96k to 160k, KV cache tried as Int8, q8v4 and q4_0, kv-resident 20480 to 32768.
Strata reached around 17 tokens/sec, 31-36 t/s for plain text generation and 20-25 t/s for logic/coding, with 5700 to 6800 experts in VRAM and system RAM maxed. Lowering kv-resident raised experts in VRAM but ran 3-4 t/s slower. User asks where the bottleneck is and which low-RAM flags to use.