Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 Flash-Next at 15.7 t/s on an RTX 4070 12GB and V100 16GB with 32GB of system RAM.
Setup is llama.cpp with IQ3_XXS weights, 98k context, flash attention, jinja, and MTP with 2 draft tokens, using layer split rather than tensor parallelism.
The user asks whether the model should require 45.8 GB in memory as shown for an AD-3.84bpw-IQ4_XS-M64 build, and notes their own RAM usage stayed around 6 GB.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8 Flash Next running on a MacBook Pro M1 Max 128GB with MTPLX at a constant decoding speed of around 30-40 tok/s.
Setup uses a 128K context window and 4 MCP servers (semble, context7, chrome-devtools, laravel-boost).
User compares the model to Claude Opus 5.5 on a simple coding task: Opus completed it in 3 minutes while Qwen took 53 minutes, with the model appearing to go in circles. Different reasoning levels (low, medium, high) yielded similar results.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User compares vLLM and TensorFold running Qwen3.8-Flash-Next on a single DGX Spark (GB10, 128 GB unified memory). vLLM with an NVFP4 checkpoint, FP8 KV cache, and MTP with 3 draft tokens reaches 27.8 tok/s for a single request, while TensorFold with a 4-bit MLX checkpoint and int8 KV cache reaches 52.2 tok/s. At 5 concurrent requests, vLLM gives 78.5 tok/s and TensorFold 110.2 tok/s. Prompt processing is 1,500-1,730 tok/s for vLLM and 2,090-2,260 tok/s for TensorFold.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8 Flash Next Q4 running through ds4 on an M3 Ultra 96GB with the full 262K context configured, delivering 55–60 tok/s decode and 667 tok/s prefill.
About 80GB is used for model, KV cache and buffers, while the model's 95GB n-gram table streams from SSD.
The user runs it daily through a codex harness as a headless box.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
agenticlong-context
User reports Qwen3.8-27B at FP8 running tensor-parallel across two Radeon AI PRO R9700s, with roughly 1.6k tokens/s prefill and 74-143 tokens/s decode single stream depending on content.
Setup is vLLM 0.27.1 in a stilldeadcode/vllm-radiance image with FP8 weights and fp8 KV cache, 262,144 context, 422,964 tokens of KV pool, and RCCL 2.27.7 replacing the image's newer RCCL because one card sits behind the chipset and cannot do PCIe atomics.
The decode figure is given as a range, so no single generationTps is recorded. The bundled P2P all-reduce is disabled (RADIANCE_USE_R4D_AR=0) and TP=2 falls back to host-staged collectives; the user notes two CPU-attached cards would keep the P2P path and get better prefill.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports running Qwen3.8 Flash-Next at q3_xss on an RTX 4080 Super with a 9800X3D and 64 GB DDR5 6000, getting 1000 t/s prefill and 40-60 t/s generation.
Setup uses the Strata engine. The user questions whether the speed increase is too good to be true and suspects the model may actually be Qwen3.6 35B A3B.
The user notes the model does not think very much and feels quite dumb.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-Distill at 165.9 t/s decode on an RTX 4090 24GB.
Setup is llama.cpp with llama-swap, q8_0 KV cache, flash attention on, 256K context, and speculative draft-MTP enabled.
The post is a registry of many models on the same card; the highest measured decode listed is 363 t/s for Nemotron-Lightning-3.5, and context cliffs are noted for several models.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codinglong-context
User benchmarks Qwen3.8-Flash-Next on an AMD Strix Halo 128GB (ASUS ROG Flow Z13 GZ302) at 70 W TDP, comparing several engines. The gufo engine with UD-Q4_K_XL weights reaches 32.9 t/s decode and 1,033 t/s prefill at 64k context (32k prompt), with 74% MTP acceptance and 14/14 retrieval.
Setup uses gufo (ROCm 7.2.4) with UD-Q4_K_XL GGUF weights and MTP speculative decoding, run via LlamaStash on Arch Linux.
Halogen 0.14.0 with native .hgn weights is fastest at 39.3 t/s decode and 1,045 t/s prefill, but is closed source and Docker-only. gufo is open source and loads 4x faster from cold. At 128k context (64k prompt), gufo gets 32.3 t/s decode and 1,047 t/s prefill; at 256k context (130k prompt), 27.9 t/s decode and 997 t/s prefill. In 10 Aider polyglot Python exercises, gufo passes 10/10 in 36.0 min, versus Halogen 21.5 min and CIRU 24.6 min.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B (Swift-1.5-HyperQwen W4A16) at 143 t/s decode and 889 t/s prefill at 32K context on 4x RTX 3080 20GB (modded) with tensor parallelism 4.
Setup is vLLM 0.29.0 with W4A16 weights, int8_per_token_head KV cache, DFlash2 speculative decoding with 7 spec tokens, max_model_len 262144, max_num_seqs 4, chunked prefill and prefix caching enabled.
Single-task decode ranges from 143 t/s at 32K to 240 t/s at 2K context; two concurrent tasks drop to 17-115 t/s. DFlash2 draft acceptance is 64.9% with 5.54 tokens/step. User previously got 40-50 t/s without DFlash2 and notes MTP degraded speeds.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B at around 33 t/s on a system with 16GB of VRAM, 32GB of RAM and a Ryzen 5 7600.
Setup is Unsloth Studio with an IQ4_XS quant.
The user asks for recommendations of other models, including ones that can generate PDF or DOCX files.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8 Flash-Next at 2466 t/s prompt processing and 167 t/s generation on an RTX 3090 plus RTX 5070 Ti with the Strata engine and IQ3_S quant.
Setup is Strata with IQ3_S; the user also added UD-Q4_K_XL support, which reached 2341 t/s prompt and 126 t/s generation.
The user doubled Strata throughput over a week of profiling and reached 5x llama.cpp on UD-Q4_K_XL, where the starting point was 6 t/s. Earlier steps were 21 t/s, 27 t/s and 51 t/s on IQ3_XXS.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B at 29.92 t/s on a mixed GPU pool of R9700, RTX 5070 Ti, and RTX 4070 using llama.cpp RPC.
Setup is llama.cpp with Q8 quant and 256k context, using MTP speculative decoding. The RPC pool runs each card on its native driver (ROCm for R9700, CUDA for Nvidias).
User compares this to Vulkan solo on the R9700 (26.48 t/s) and their daily driver Q6_K_XL at 131k context (59.08 t/s). RPC showed tighter consistency (spread 7.4 vs 16.3). Flash Next saw no improvement.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8 27B at 75 t/s decode on 2x RTX 5060 Ti (32GB total VRAM), with a peak of 120 t/s.
Setup is vLLM with NVFP4 weights, serving 2 concurrent requests at 130k context.
User also tried Swift 1 and 1.5 with no noticeable improvement and a context penalty, and finds the model's coding output mediocre for bounded tasks.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User describes a Swift-based LLM harness built around Qwen 3.8 Flash-Next and Qwen 3.8 27B that adds a runtime slider to suppress prompt refusals without reloading or swapping to a separate abliterated model.
The slider trades model coherence for reduced refusal, and lets the main agent spawn abliterated sub-agents. No hardware, engine, quant or context length is stated.
MTP acceptance drops as the slider is raised, taking decode from 60-70 tok/s down to 20-30 tok/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
agenticcodinglong-context
User benchmarks Qwen3.8-Flash-Next 8-bit on a Mac Studio M5 Ultra 256GB, reporting 17 t/s decode and 1,069 t/s prefill at 250k context on llama.cpp (Metal).
Setup is llama.cpp with Q8_0 weights and MTP off; oMLX 0.7.0rc1 reaches 51 t/s decode and 1,365 t/s prefill at the same depth, and 88 t/s decode at 10k with MTP on.
A cold 250k-token prompt took 234 s before the first token, while the cached next step took 0.64 s. DeepSeek-V4-Flash at 400k and 500k context took 934 s and 1,311 s cold. The user notes one overnight agent run per setup and that the DeepSeek engine comparison mixes two model variants.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8 27B Swift at 143 tok/s decode and 2840 tok/s prefill on an RTX 5090.
Setup is llama.cpp with Q6_K quantization.
The figures come from a coding prompt asking for an HTML/JS calculator, which produced a working result in 1 minute 10 seconds; the user calls this the ceiling after two days of trials.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagentic
User benchmarks a local Qwen3.8-27B at 52.0 tok/s decode and 708 tok/s prefill on an RX 7900 XT, used as a coding executor for Claude across 51 runs.
Setup is llama.cpp llama-server with IQ4_XS at 64K context, q8_0 KV cache, flash attention, full GPU offload, and MTP speculative decoding (draft n_max 2, p_min 0.75).
The 27B was correct after review on 43/51 runs versus Sonnet 5's 45/51, tied on full-code and debug tasks, and failed all interface-only lifecycle and in-memory repository tasks. End-to-end throughput in the agent was 47.4 tok/s for the 27B and 107.8 tok/s for Sonnet, which cost $9.20 in API usage.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8-27B at 115 t/s decode on a single RTX 4090 24 GB.
Setup is llama.cpp b11115 with llama-swap v257, IQ4_XS GGUF quant, q8_0 KV cache, 196608 context, and draft-MTP speculative decoding.
The user notes the DeepSWE result is a single task, not an average, and that the model passed 40/43 hidden tests and kept 109/109 existing tests.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingagentic
User reports Qwen3.8-27B generating an admin-panel application at around 35 TPS, finishing the initial implementation in about 4 minutes and 30 seconds.
No hardware, engine, quant or context length is stated.
The user says the result is far from production-ready and estimates roughly 500 bugs remain.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
agenticcoding
User compares NInfer's official Qwen3.8-27B quant (part NVFP4, part FP8) against QUASAR's QAT full-NVFP4 checkpoint with DFlash2 embedded, both on a single RTX 5090.
QUASAR reaches 484 tok/s on JSON output, 204 tok/s on prose, and 13.7k tok/s prefill, using 27.0 GB VRAM. The official quant gets 400 tok/s JSON, 176 tok/s prose, 11.4k tok/s prefill, and 30.8 GB VRAM.
On a real agentic task the QUASAR build averages 287 tok/s over 33k tokens. GPQA Diamond scores 89.9% versus 90.4% for the official quant, and perplexity is about 1.9% worse.