Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8 Next Q3_K_XL at 24/27 tk/s on prose and 40+ tk/s on code with MTP on two 7900 XTX cards, with prefill at 920 tk/s pp8192.
Setup is a custom llama.cpp build optimized for the 7900 XTX, with RAM offload and without the MoE expert cache.
The user also benchmarks Qwen3.8 27B Q8_0 with tensor parallel at 1600 tk/s pp8192, 60/65 tk/s prose and 100+ tk/s code, and Qwen3.6 27B Q4_K_M on a single card with MTP at 1020 tk/s pp8192, about 58/60 tk/s prose and 75/80 tk/s code.
User reports a task-aware quantization (TAK) of Qwen3.8-27B scoring 82.81% on a reasoning benchmark, against 77.34% for Unsloth UD IQ2_S and 83.59% for BF16.
No hardware or engine is mentioned, and no quant code is given for the TAK variant.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
coding
User switched from Qwen 3.6 27B to Qwen 3.8 Next Flash.
User reports very verbose output and long thinking times, including 13 minutes on single-turn coding requests.
User uses pi.dev with BYOK to VSCode and criticizes the output quality for decision-making tasks, calling it alphabet soup.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagentic
User reports Qwen 3.8 27B IQ3_K_XXS by Unsloth running fully on a 4060Ti 16GB at ~100k context, averaging 800 t/s prefill and 20 t/s decode.
Setup uses a Q8 KV cache with mmproj and MTP dropped.
Decode falls to 17 t/s after 64k context. The user runs agentic coding with parallel tool calls and successfully merged a feature branch, and plans to upgrade to an R9700.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
agentic
User compares an M5 Ultra 96GB against an M5 Max 128GB for running Qwen3.8-27B at Q8, and reports about 15 t/s on the Ultra.
The user also discusses Qwen3.8-Flash-Next, a multimodal MoE with 176B total parameters and about 6B active, and estimates it will not fit in 96GB.
MLX and llama.cpp are mentioned as possible engines, but the user does not confirm using either.
User benchmarks vision models on calorie estimation from meal photos. Qwen 3.8 27b scores 16% within 20% error, with a mean bias of +64 kcal and a median error of 148 kcal.
Other models tested are GLM 5.3 Flash, Qwen 3.8 Max, Muse Glimmer 30b, Qwen 3.8 Flash, DeepSeek v4 Flash Vision, and Muse Spark 1.3.
The user notes that model size does not correlate with performance, and that the best model on consumer hardware (~32GB VRAM) depends on the task.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 2.4T at 0.25 t/s on dual RTX 5090s and three RTX 3090s with 96 GB of system RAM.
The model is 397 GB, far exceeding available VRAM and RAM, at 64k context.
The user deems the speed unusable.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-2.4T-A95B at ~0.80 tok/s generation on an RTX 5090 plus an RTX 5060 Ti.
Setup is llama.cpp (Unsloth build 10360) with the Unsloth Q1_0 GGUF (397 GiB), native MTP speculative decoding at n_max=3 and p_min=0.5, and 92 experts offloaded to CPU.
MTP acceptance is 90.48%, a +3.64% throughput gain over no MTP. VRAM use is 29.6 GB on the 5090 and 12.3 GB on the 5060 Ti.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen 3.8 max (preview) generating at 60-65 t/s.
User praises its creativity and writing but criticizes excessive thinking time and hallucination.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User asks about an MTP or DFlash head for Qwen 3.8 27B, and mentions 35B-A3B as a daily driver.
User reports about 8 tok/s for Qwen 3.6 27B with MTP on 32 GB of unified memory.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B at about 200 t/s generation on a single RTX 5090 with speculative decoding, on day-0 support in NInfer.
The setup uses a shared paged KV cache and ReplaySSM for GDN, with PDL in use.
The user notes support for up to 8 concurrent requests.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User compares Qwen3.8-27B-FP8 against Qwen3.6-27B-FP8 on an RTX PRO 6000 Blackwell with vLLM.
In an MTP sweep, Qwen3.8 peaks at MTP 6 with 114.8 tok/s and Qwen3.6 peaks at MTP 7 with 131.6 tok/s.
Qwen3.8 is 5-20% slower across MTP steps, though quality is comparable or better at some steps. A pre-sweep manual run at MTP 5 showed Qwen3.8 faster, at 108.9 vs 103.6 tok/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8-27B at around 4.5 tok/s on an RTX 5070 Ti Laptop GPU with 12GB VRAM and 32GB system RAM.
Setup is Unsloth Studio with UD-Q4_K_XL (roughly 17.9GB), 8K context, Q8 KV cache, flash attention, one parallel slot and MTP speculative decoding with draft max 2. Around 10,978 MiB stays in VRAM and the rest is offloaded to the CPU.
A longer response ran at 3.26 tok/s, a short factual one at 4.42 tok/s and a Python coding one at 4.53 tok/s, with prompt processing around 20-27 tok/s. Draft-token acceptance was around 78-83%.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B at over 200 t/s decode on an RTX 5090 and an RTX Pro 6000, and 38 t/s on a DGX Spark.
Setup is NVFP4 with DSpark on day-0 support.
The user also mentions an H200 in the cookbook link.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports 34-36 t/s on an RTX 3090 with Qwen3.8 27B.
Setup is llama.cpp with the IQ4_NL quant and a Q8 KV cache.
The user compares this to 50-60 t/s on Qwen3.6 27B.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
creative-writing
User compares Qwen3.8-27B against Qwen3.6-27B on an RTX 5090.
The user reports prefill and decode speeds at concurrency 1, and finds the subjective quality improvement smaller than benchmarks suggest.
Medium reasoning used fewer tokens than low on 3 of 5 prompts. The concurrency knee sits at 3 requests for Qwen3.8 versus about 7 for Qwen3.6.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports about 40 t/s with a Q4_K_XL quant on 2x RTX 3060 12GB.
Setup is llama.cpp with flash attention and speculative decoding.
The model wrote CUDA code but could not run it due to VRAM limits.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
summarization
User compares Qwen3.8-27B against Qwen3.6-27B on a fact-extraction task, where Qwen3.8 scores 0.7030 F1 against Qwen3.6's 0.7177, a statistical tie.
Decode throughput falls from 85.6 to 72.1 t/s, about 16%, under the closest saved configs, though the runs used different llama.cpp builds.
Qwen3.8 produces shorter answers, lowering end-to-end latency. The user expected larger gains from public benchmarks, and sees small gains on task-specific tests but massive gains only on benchmarks the model was trained on.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8-27B at 73 t/s on code and 58.5 t/s on prose on dual RTX 3090s (24 GB each, 48 GB total).
Setup is a Q8_K_XL quant (31.5 GB) with mmproj-F16 (0.93 GB), 200K context and an f16 KV cache, tensor split 50/50. MTP draft depth 2 gives the best acceptance at 92% on code and 68% on prose.
The speeds are medians of 3 draws at temp 0. The hybrid architecture has 48 of 64 layers as Gated DeltaNet (linear attention), with only 16 full attention layers plus the MTP head, for 17 KV-caching layers. Cold start is 32s. The split is ~1.1 GB lopsided, causing OOM risk at 220K context. User asks for optimization suggestions for dual 3090 setups.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B at 70.19 tok/s end-to-end (71.00 tok/s decode) for a single request on an RTX 3090.
Setup is the RTX 3090 port of NInfer, a C++/CUDA inference runtime, with MTP3 speculative decoding at 61.13% acceptance, CUDA Graphs enabled and 1,024 output tokens per request, peaking at 19,641 MiB VRAM.
At eight concurrent requests decode reaches 165.33 tok/s. Qwen3-35B-A3B runs at around 260 tok/s single-stream on the same setup, and over 400 tok/s on highly repetitive workloads.