Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8 at 118.6 t/s on two NVIDIA DGX Spark (GB10 Grace Blackwell) machines at TP=2.
Setup is vLLM with NVFP4 experts and MXFP8 dense/attention paths, FP8 KV cache, probabilistic MTP with 4 draft tokens, and a configured max_model_len of 262144.
The figure is the mean of 3 boot-level means of per-pass median request decode rates over 36 coding prompts run one at a time, with a between-boot sample SD of 0.353 t/s; the workload is favorable to MTP and is not a general conversational speed estimate.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8 at 78.6 t/s on one NVIDIA DGX Spark (GB10 Grace Blackwell, 128 GB unified RAM, TP=1).
Setup is vLLM with NVFP4 experts and MXFP8 dense/attention paths, FP8 KV cache, probabilistic MTP with 4 draft tokens, and a configured max_model_len of 262144.
The figure is the mean of 6 boot-level means of per-pass median request decode rates, one request at a time, with a between-boot sample SD of 0.487 t/s; it is not pooled tokens/time or an end-to-end completion rate.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codinglong-context
User reports running Qwen3.8 Flash-Next IQ3_XXS at a 700K context window on an RTX 5090 32GB with 64GB system RAM, completing a project in one continuous session without context compaction.
Generation speed is given as approximately 100-120 tokens/sec, with vision disabled. The model ran with CPU offload given the 125B size on a single 32GB card.
User notes possible quality degradation at extreme context lengths and estimates 1M context might be possible at around 80 tokens/sec, though untested.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingagentic
User benchmarks five quants of Qwen3.8-27B on 2x RTX 5070 Ti 16GB, finding standard evals (GSM8K, MMLU-Pro, IFEval) cannot distinguish them within ±2 points. The UD-Q4_K_XL quant reaches 124 t/s decode with MTP and 245,760 tokens of context, versus 93 t/s and 32,768 tokens for Q8_0.
Setup is llama.cpp with tensor split, MTP draft and q8_0 KV cache; NVFP4 runs use vLLM TP=2. Q4_K_XL picks a different token than Q8_0 once every 28 tokens by top-1 agreement.
On the user's own agentic coding bench (112 runs per model, OpenCode, hidden tests), 27B NVFP4 solves 77% and Flash-Next NVFP4 98%; on the 6 hardest tasks it is 1/12 versus 12/12. SWE-rebench agrees at 69% versus 82% (paired p ≈ 0.007). Flash-Next Q3 on Strata matches NVFP4 on 11 of 12 discriminating tasks but always misses one causality bug.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
summarizationvisionlong-context
User reports Qwen3.8 27B at 14.0 tok/s generation and 93.1 tok/s prompt processing on a Mac Studio M3 Ultra with 256GB unified memory.
Setup is Ollama with Q4_K_M (17GB) and five timed runs averaged; the predecessor qwen3.6:27b reached 28.6 tok/s on the same machine, and the 1-bit UD-IQ1_M quant (6.7GB) ran at 27.2 tok/s generation and 309 tok/s prompt processing in llama.cpp.
User notes the new hybrid attention architecture is slower per token but uses roughly a third the tokens per answer, so wall-clock per answer is close to a tie; community figures cited for Strix Halo (20.4 tok/s), dual RTX 3090 (~60 tok/s) and RTX 5090 (75-140 tok/s) are not the author's own measurements.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagentic
User reports Qwen3.8-27B at 53.3 tok/s per-request decode (47 to 58 across eight requests) on an Intel Arc Pro B60 24GB.
Setup is llama.cpp Vulkan build b11524 with Q4_K_M and an F16 KV cache, a DFlash2 draft model, one request at a time, 32K context, the whole model on the card.
The repo's 42.3 tok/s headline is wall-clock eval-suite throughput with prompt processing included, not a decode rate; stock llama.cpp with no drafter gave 16 tok/s in llama-bench tg32. Prompt reading at 32K was 420 tok/s with the Intel FA kernel, versus 118 tok/s stock.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next at 100.2 tok/s on an M5 Ultra Mac Studio with 256GB unified memory.
Setup is llama.cpp b11538 with IQ4_NL weights, f16 KV cache, 262K context, and MTP speculative decoding with probabilistic drafting at n-max 5.
Without speculation the model decodes at 57.0 tok/s; prompt processing reaches 2150 tok/s with ubatch 4096. The full 262K context fits with about 103 GB of GPU memory free. A second copy of the model loaded by another app caused swapping and dropped speed to 12 tok/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks Qwen3.8-Flash-Next oQ6e at 135.2 t/s decode and 2,458 t/s prefill on an M5 Ultra 256GB, comparing oMLX 0.7.0 and 0.7.1.dev1.
Setup is oMLX with MLX oQ6e weights, 8-bit KV cache, Lightning MTP on, 1K context, single request.
Across 4K-200K contexts, average decode is 120.9 t/s and prefill 4,761 t/s; 2 parallel requests give 74.2 t/s per request and 4 parallel 41.0 t/s per request. GLM-5.3-Flash oQ4 reaches 97.9 t/s decode and 2,264 t/s prefill, and Flash-Next oQ4e 121.6 t/s decode and 5,450 t/s prefill.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B running at 10-20 tok/s on an M5 Pro Mac Mini with 48GB RAM, below the 30-50 tok/s they expected from community optimizations.
Setup is oMLX 0.70 with both 4-bit and 8-bit models; the 4-bit run took 40 minutes on a lengthy command and the 8-bit run progressed only 5% in over 10 minutes before being cancelled.
The user asks whether this performance is expected for their hardware.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
agenticlong-contextvision
User reports Qwen3.8-Flash-Next running on a BOSGAME M5 with AMD Ryzen AI Max+ 395 / Radeon 8060S (Strix Halo) and 128 GB unified RAM, showing ~69 tok/s peak decode and ~1,155 tok/s prefill with peaks around 1,685 tok/s during a real long-running agent workload.
Setup is Halogen 0.17.2 on Ubuntu 26.04 with 262K context configured and vision enabled; decode stays in the 60–70 tok/s range while prefill remains above 1k tok/s for much of the workload.
The figures come from a Grafana dashboard tracking Halogen throughput, KV/context usage, RAM, PSI memory pressure and GPU metrics; context was ~84% full, 82K tokens generated in the last hour, GPU around 62°C and 24 W, with ~121 GB physically occupied but mostly reclaimable cache (~85 GB).
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8 Flash-Next at 60.39 t/s generation and 1700.52 t/s prompt processing on a Strix Halo 128GB device (GPD Win5 128G).
Setup is the Gufo engine with Q4_K_XL quant, single user.
The same engine also ran Qwen 27B Q4_K_XL at 70.56 t/s generation and 656.33 t/s prompt with DFlash2, and served 8 users at 162.98 t/s aggregate. User says performance is close to Halogen and better than llama.cpp variants on prefill, and is switching to it for daily use.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next (FP8) at 15.0–17.2 tok/s decode and 500–600 tok/s prefill at 16K context on an M5 Pro 64GB Mac.
Setup is Whallm 1.1.11 with MTP on by default, streaming experts from SSD; peak MLX memory stays under 19 GiB at 16K context.
A second row for Swift1.5-Qwen3.8 (MXFP4) gives 14.6–17.5 tok/s decode and 571 tok/s prefill at 18.8 GiB peak. Previous v1.1.7 numbers were ~9 tok/s decode and ~146 tok/s prefill.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B INT4 at 105.9 tok/s single-stream on 4x Intel Arc Pro B65 32GB with tensor parallelism 4 and MTP-2.
Setup is vLLM 0.31 with RedHat INT4 weights, 200 W cap per card, PCIe 4.0, on an AMD EPYC 7343 host with 499 GiB RAM.
At c32 the four-card setup reaches 712 tok/s aggregate; four one-card replicas reach 831 tok/s at c32. Prefill is 3,809 tok/s on an 8K prompt and 2,476 tok/s on 128K. Quality is +4.29% perplexity versus BF16.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Hauhau's Qwen3.8 27B Uncensored at around 20 tok/s on a single RTX 3090 with MTP enabled.
The base Qwen model outputs around 30-35 tok/s on the same card.
User asks why the uncensored fine-tune is substantially slower than the base model.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Mirai S, a fine-tune of Qwen3.8 27B, at 70 t/s with 262k context on a 12GB GPU.
Setup details are not stated; the post describes a 12GB+ tier getting full 262k context and an agentic improvement patch, with 16GB+ users getting all positions in VRAM for better numbers.
The user says Mirai S retains 98% of the original Qwen3.8 27B capabilities and calls it a strong result for a 12GB Facebook Marketplace GPU.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
agenticcoding
User reports quantizing ukisai's Swift 1.5 Qwen3.8-Flash-Next to NVFP4 for a 2x DGX Spark pair running stock dgpp-serve with TP=2. The 126 GiB checkpoint runs at ~55-60 tok/s C1 and ~114 tok/s C4 on prose, with prefill ~1.9-2.3k tok/s.
Setup is dgpp with NVFP4 weights, 126 GiB, 296k tensors, 52 shards, no engine changes.
The fine-tune is a reasoning-efficiency variant that cut output ~24% (805 to 611 tokens) on a chat probe with no speed loss, but savings are prompt-dependent and one zebra puzzle went up 39%; the upstream 31-57% thinking-token claim was not reproduced on a real eval suite.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingagenticvision
User reports Qwen3.8-27B at ~5.99 tok/s with MTP speculative decoding on an RTX 4060 8GB with 32GB system RAM.
Setup is llama.cpp with UD-Q4_K_XL GGUF at 16K context, one parallel slot, partial CPU offload since the 17.56GB model exceeds 8GB VRAM. MTP accepted 99 of 126 draft tokens (~79%), improving decode by roughly 57% over the ~3.82 tok/s without MTP. The IQ4_XS quant measured ~4.17 tok/s.
User also ran Qwen3-VL 4B Q4_K_M invoice extraction: 5 of 6 image-only invoices completed in 8-38 sec, and 4 of 6 with OCR transcripts in 25-40 sec. In a 2-invoice comparison Qwen matched 22 of 24 top-level fields versus Gemma's 5 of 24. User notes these are application-level results, not standardized benchmarks.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Swift-1.5-Qwen3.8-27B-GSQ-RCO 3 xxs running on an RTX 5060 Ti 16GB with 4-step MTP and a 100k context at Q8 KV cache, leaving around 300MB of VRAM free.
Performance is about 38-60 t/s, stabilizing around 45-50 t/s on long tasks; without MTP the model runs at 22-30 t/s.
User says the quant is faster and better than the previous Swift model, with code quality close to Q4 performance.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8-Flash-Next UD-Q4_K_XL on an RTX PRO 6000 with Strata, INT8 KV cache at 256K context, and MTP-4 speculative decoding. Single-request decode varies by task: 185.8 t/s prose, 320.4 t/s counting, 298.9 t/s coding, 289.1 t/s reasoning, with prefill at 3,319 tok/s for a 17.3K prompt. vLLM on the same GPU with NVFP4 quant reached 95.3-171.0 t/s decode and 4,084 tok/s prefill, while vLLM on DGX Spark TP2 reached 38.2-75.3 t/s decode and 2,825 tok/s prefill.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User asks whether Qwen3.8 27B Q8_0 numbers on a W7900 are reasonable.
Table gives mean prefill 982.2 tok/s, mean generation 54.9 tok/s and mean server TTFT 2.27 s at depth 0, with 32K and 128K rows kept in features.
No engine, context length or KV cache quant stated.