Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports about 920 tokens/s prompt processing for Qwen3.8 Flash-Next MoE UD_Q3_XXL on two RX 7900 XTX cards, and about 1,600 tokens/s prompt processing for Qwen3.8 27B dense Q8_0 with tensor parallel.
Setup is a custom llama.cpp RDNA3 fork with ROCm 7.14, adaptive MTP draft depth up to 3, Q8_0 KV cache, layer split, and MoE experts offloaded to system RAM with a 288-slot GPU-resident expert cache.
The user notes the expert cache slows prompt processing to a crawl on their workload and they run with it disabled; with 288 experts they report a 95% hit rate and 45 t/s decode with MTP. The build is experimental and the figures are measured, not guaranteed.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8 Flash-Next at 39.8 t/s mean decode on two DGX Sparks, with final code decode at roughly 74 t/s mean.
Setup is vLLM with FP8 weights and NVFP4 KV cache, TP=2 across two DGX Sparks, DSpark speculative decoding with 3 proposed tokens, single concurrency.
User also tried DeepSeek V4 Flash on the same setup but it failed benchmarks due to non-completion and excessive token use (30-45k tokens on an easy task), unlike cloud versions of the same model. User is happy with Qwen3.8 Flash-Next but concerned about overthinking tendency.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8 at 210.5 t/s decode and 6708.5 t/s prefill on a single RTX 5090.
Setup is NInfer with MTP speculative decoding, 1690 prompt tokens and 3029 generated tokens, 254 ms TTFT.
The user runs a demoscene generator where the model writes Three.js effects and revises them from captured video; LM Studio is also supported for the regular workflow.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next at 1700.52 t/s prompt processing and 25.95 t/s generation on an AMD Strix Halo 128GB.
Setup is Gufo with an Unsloth UD-Q4_K_XL GGUF, single-user autoregressive with no speculative decoding, greedy sampling, thinking off, at cached depth zero (prompt_n 2040, predicted_n 128).
This is an engine-maintainer measurement, not an independent reproduction, and the measured binary was built from a dirty source tree. Allocated context capacity is 133760 tokens, not prompt length. MTP, aggregate concurrency and cached-depth rows are separate workloads.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8 27B at 10 t/s decode on a Strix Halo 128GB using gufo, the same speed they saw with llama.cpp.
Setup is gufo with default parameters except the thinking spec, on a software design task that required reading an existing repository.
The user says the model did great work but took 90 minutes where a frontier model took 5, and asks how to speed the task up.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 Flash-Next Abliterated at 110 tok/s on an RTX Pro 6000 Blackwell 96GB.
Setup uses Q5 or Q6 quantization with 128k context.
The user rents the GPU monthly and only runs it during the day, leaving it idle for about 8 hours each night.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next at ~64 tokens/s single-stream decode and ~1,200 tokens/s prefill on four AMD Radeon PRO V620 cards (32 GB each, gfx1030) in one box.
Setup is a vLLM fork with AWQ W4A16 backbone plus a quantised int4 n-gram table, tensor parallel across all four cards, MTP off and cards uncapped; the n-gram table is CPU-offloaded.
At the production 140 W setting decode is ~59.5 tokens/s, and with MTP=3 it ranges ~60-72 tokens/s. Model files total ~105 GB of downloads.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Swift-Qwen3.8-27B at 24.6 t/s decode and 492 t/s prefill on a single Radeon PRO V620 32GB.
Setup is a llama.cpp fork for gfx1030 with Q4_K_XL GGUF (16.6 GiB), flash attention, 128k context, card at 300 W.
Decode falls to 16.9 t/s at 128k depth; with an MTP drafter it reaches 48.0 t/s at depth 0 and 44.1 t/s at 120k, while DFlash2 peaks at 76 t/s on short mixed work.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codinglong-contextagentic
User reports Qwen3.8-27B EXL3 4.00bpw at 91.2 t/s decode (C1, prose) on one Intel Arc Pro B70 32GB.
Setup is vLLM 0.26.1 XPU with EXL3 trellis quantization, fp8 KV cache, 262,144 context, MTP speculative decoding k=3, int8 prefill and oneDNN fused attention.
Aggregate throughput scales to 365.2 t/s at C16; cold prefill is 2,421 t/s at 4K and 911 t/s at 254K. The same card with tuned llama.cpp SYCL Q4_K_M reaches 25.0 t/s at C1 and 999 t/s prefill at 4K.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
agentic
User reports Qwen 3.8 27B at up to 42 t/s on 2x Radeon Pro V620 32GB cards at 41k context.
Setup is vLLM 0.27.1 with a Q4 quant and tensor parallelism of 2.
The first working run gave 11-18 t/s; after optimization the two-card combo reached 42 t/s. A four-card TP=4 config for a 122B MoE reaches 56-59 t/s. A successor repo runs Qwen3.8-Flash-Next at ~100 t/s single-stream on 4x V620, versus 29-30 t/s for llama.cpp on the same box.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next at 62-68 tok/s decode on 3x Radeon PRO V620 32GB in pipeline parallel PP=3.
Setup is vLLM with an AWQ-W4A16 checkpoint, int4 n-gram sidecar in host RAM, MTP k=2 speculative decoding, 262K context, 160 W power cap per card.
Prefill measured at 1152 / 1705 / 1936 / 1933 tok/s for 4K / 16K / 64K / 130K prompts; a 200 W cap raises prefill to 1306 / 1943 / 2221 / 2254 tok/s with decode unchanged. Multi-stream decode without MTP reaches 157 tok/s at 4 streams and 242 at 8; with MTP 159 at 4 and 91 at 8.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingtool-use
User reports Qwen3.8 Flash-Next at 12 t/s on an RTX 4070 Super with 80GB DDR4. Setup is Unsloth desktop, but the model errors out on anything beyond simple questions and refuses coding tasks with a message format error.
The user also tried Qwen 27B in Q2 at 3 t/s, Qwen 3.6 35B at 20 t/s, and Ornith, which thinks for 30 minutes before erroring. They mention using LM Studio, Ollama, Open WebUI, and llama.cpp in the browser, and note that Qwen 27B with Q4 KV cache at 8K context gives 6 t/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8 27B at about 30 tok/s on an RTX 5080 with two RTX 5060 Ti 16GB cards in a multi-GPU setup.
Setup is LM Studio with the default Qwen parameters; the two secondary cards run at PCIe 4.0 x4.
The user asks whether swapping in an RX 7900 XT or switching to Vulkan would improve speed, and whether a Thunderbolt 3 eGPU would only affect model loading time.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B at about 50 t/s decode on a V100 32GB.
Setup is llama.cpp with UD-Q4-K-M (MTP) and batch size 4096, ubatch 2048, on a PG500-216 card at 185W with a custom compile.
User notes prefill drops off quickly and asks for pointers on improving it.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8-27B at a stable 22 t/s generation with up to 95k context on an Intel Arc Pro B60 24GB connected via Oculink to a GMKTEC M7 Pro with 32 GB RAM.
Setup is intel-llm-scaler with an INT4 quant. The user also tried a Swift-Qwen3.8 fine-tune and MTP, but MTP limited context to about 50k and produced garbage quickly. llama.cpp was slower than vLLM on both Vulkan and SYCL.
User asks whether better options exist for a 24 GB card than Qwen3.8-27B, mentioning Qwen3.8-Next-Flash as likely too heavy.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next at 60 to 100 t/s decode and over 3000 t/s prefill on a rig with 8x Radeon Pro V620 (256 GB VRAM).
Setup is a custom vLLM fork with W4A16 routed experts and BF16 elsewhere, pipeline parallel 4, MTP enabled with 3 token drafting, concurrency 1.
User says llama.cpp managed only around 350-450 t/s prefill with the same model, and that the new setup is about 800% faster.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B abliterated MLX 4-bit at 33.374 tok/s median C1 decode with DFlash 2 block 3 speculative decoding on an Apple M5 Max.
Setup is MLX 0.32.1 with MLX-LM 0.31.3 and MLX-VLM 0.6.16, affine 4-bit group size 64, 256-token prompt and 256 fixed generation tokens, greedy decoding, one warmup and two timed trials.
Autoregressive C1 median was 27.011 tok/s, giving a 1.2356x speedup from DFlash 2. Peak memory was 18.435 GB with the 4-bit DFlash draft and 16.062 GB in the quality suite. The 33.374 tok/s figure is a same-run M5 Max C1 measurement, not a guarantee for every Apple chip or thermal state.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
agenticvisionlong-context
User reports Qwen3.8-Flash-Next at 21.0 tok/s on an Intel Arc B580 12GB.
Setup is Strata v0.1.40.4 with IQ2_XS quantization and INT8 KV cache, 256K context, 2808 resident expert slots, speculative decoding with MTP.
Long-context stress test at 74,213 prompt tokens sustained 17.8 tok/s for 237.7 seconds. User compares to llama.cpp/Ollama offload at 1.5-3.0 tok/s and previous Strata version at 17.1 tok/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B at 139.7 tok/s on an RTX 5090 with SGLang and a D-Flash 2 drafter.
Setup is SGLang with NVFP4 weights and an FP8 KV cache, single-user at concurrency 1. Other profiles on the same card: DSpark v2 drafter 110.2 tok/s, native MTP under vLLM 100.8 tok/s, and a vLLM baseline with no speculation at 84.5 tok/s.
Prefill is about 9,081 tok/s on the 5090 against about 196 tok/s on an M4 Max 128GB running oMLX 4-bit MLX builds, where decode was 16.7 tok/s with speculation off, 35.5 tok/s with DFlash 2 greedy and 47.2 tok/s with DFlash 2 thinking on. A warm prefix cache cuts time to first token by about 90% on both machines.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8-27B Uncensored at 25.48 tok/s with DFlash2 speculative decoding on an M4 Pro 48GB, a 2.06x speedup over the 12.35 tok/s baseline.
Setup is dflash 0.1.0 on MLX with a 4-bit MLX target model and a 4-bit draft, block size 5, greedy decoding on HumanEval with 20 samples and 256 max new tokens.
Mean acceptance length was 4.87 tokens with 94.9% of blocks accepted in full. An initial 3-sample smoke test gave 13.76 tok/s baseline and 29.11 tok/s with DFlash. Weights and caches were on an external HDD; the 825.84s wall time includes model loading and is not generation throughput.