Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports 34-36 t/s on an RTX 3090 with Qwen3.8 27B.
Setup is llama.cpp with the IQ4_NL quant and a Q8 KV cache.
The user compares this to 50-60 t/s on Qwen3.6 27B.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8-27B at 73 t/s on code and 58.5 t/s on prose on dual RTX 3090s (24 GB each, 48 GB total).
Setup is a Q8_K_XL quant (31.5 GB) with mmproj-F16 (0.93 GB), 200K context and an f16 KV cache, tensor split 50/50. MTP draft depth 2 gives the best acceptance at 92% on code and 68% on prose.
The speeds are medians of 3 draws at temp 0. The hybrid architecture has 48 of 64 layers as Gated DeltaNet (linear attention), with only 16 full attention layers plus the MTP head, for 17 KV-caching layers. Cold start is 32s. The split is ~1.1 GB lopsided, causing OOM risk at 220K context. User asks for optimization suggestions for dual 3090 setups.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B at 70.19 tok/s end-to-end (71.00 tok/s decode) for a single request on an RTX 3090.
Setup is the RTX 3090 port of NInfer, a C++/CUDA inference runtime, with MTP3 speculative decoding at 61.13% acceptance, CUDA Graphs enabled and 1,024 output tokens per request, peaking at 19,641 MiB VRAM.
At eight concurrent requests decode reaches 165.33 tok/s. Qwen3-35B-A3B runs at around 260 tok/s single-stream on the same setup, and over 400 tok/s on highly repetitive workloads.
User benchmarks Qwen3.8-27B on 4x RTX 3090 (24 GB each) with vLLM 0.20.1, reaching 587.4 tok/s at c32 with TP=2 on one NVLink pair versus 417.9 tok/s with TP=4.
Setup uses tensor parallelism across the cards, with a KV pool of 1.45M tokens on TP=4 and 503k tokens on TP=2, and a max context of 262k.
MTP speculative decoding is slower at all concurrency levels, and pipeline-parallel throughput stays flat with concurrency.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B at about 124 t/s greedy on a single RTX 3090.
Setup uses fp8 KV cache, int8 lm_head and embed_tokens, MTP-4 drafts, GPTQ-int4 lm_head and MTP module, a split-KV attention kernel, and KVarN 4/2-bit KV cache for 262k context.
Peak concurrent throughput is about 1000 t/s at 64 concurrent.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B at 82 t/s single request on an RTX 3090, with up to 672 t/s peak and 417 t/s sustained at 64 concurrent.
Setup is vLLM with W4A16 quantization, an FP8 KV cache, and int8 lm_head and embed_tokens, at up to 195k context.
The title says Qwen3.8-27b while the body mentions Qwen3.6-28B.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
creative-writing
User reports Qwen3.8 27B at about 40 t/s for SVG generation on 2x RTX 3090.
Setup is the cyankiwi/Qwen3.8-27B-AWQ-INT4 model with TP2 and an FP8 KV cache.
Speed started near 60 t/s and dropped to the 20s after the user removed a power limit mid-generation. The run took 20 minutes, 12 of them thinking, and the user was impressed with the result.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B-heretic-ara Q5_K_M GGUF at ~30-32 t/s on an RTX 3090 with 64 GB DDR4 RAM and an AMD 7950x CPU.
Setup is llama.cpp.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen 3.8 27B at 100+ t/s generation at high context on 4x RTX 3090, each throttled to 200W.
Setup is vLLM with the flashinfer backend, INT8/W8A16, and MTP.
User adds that Gemma models also benefit, but vision models need --language-model-only.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B at 99 t/s single request and 1150 t/s batched on an RTX 3090.
Setup uses a hyper-optimized inference engine with an fp8 KV cache, int8 lm_head and embeddings, fp16 recurrent state, and int8 activations, at 64k context in fast mode.
Prefill reaches 1812 t/s at 1k context and 1000 t/s at 100k context.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports 120.1 t/s decode on narrative text and 218.3 t/s on code with vLLM v0.26.1rc1 and AutoRound INT4 (group 128) on 2x RTX 3090 over PCIe Gen4 x16/x16 with no NVLink and patched P2P.
Prefill reaches 1342 tok/s at 10k and 628 tok/s at 90k. Speculative decoding uses 7 draft tokens with an acceptance length of 3.35 and 47.8% acceptance.
Peak VRAM is 22.3 GB per card with zero leak, and the context ceiling is 131k because the DFlash2 drafter takes about 13.5 GB. Power is capped at 220/250 W, and the user used Kimi K3 for vLLM fixes.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B at 55 tok/s on an RTX 3090.
Setup is the IQ4_XS quant.
The user also mentions GLM-5.3 pricing and intelligence, DeepSeek V4 Flash, and Strix Halo hardware, but does not run them.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3-235B-A22B at about 7.5 t/s generation and about 75k pp on a single RTX 3090 with 128 GB of system RAM.
Setup is textgen with batch size 1024, 16384 context, autofit layers, and 38 of 48 threads on a dual Xeon DDR4 system.
Generation dips to about 7 t/s at 10k context. The user calls it a daily driver for months for uncensored ERP roleplay, and mentions Step 3.7 Flash as an alternative that is censored and verbose, along with Gemma 4 and MiniMax-M2.5, which were not run.
User reports Qwen3.8 27B Instruct scoring 36 and 34 composite on two ACT practice tests, with perfect reading scores.
Setup is 2x RTX 3090 with full offload and 32k context.
The two tests took 177 minutes in total.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
agenticcoding
User reports 50-65 t/s generation on dual RTX 3090, sometimes dropping to 40s.
Setup is llama.cpp with the Qwen3.8-27B-UD-Q6_K_XL GGUF, --mmproj for vision, --kv-unified, --flash-attn, and q8_0 KV cache types, plus speculative decoding with draft-mtp.
The use case is agentic coding.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B at 382 t/s on an RTX 3090 when reproducing context, and about 133 t/s on real chat prompts.
Setup uses an fp8 KV cache, int8 activations, DFlash2 block drafting, lookup-augmented drafting, prefix caching, split-KV verify attention, and KVarN for 262k context.
Quality is unchanged, with GSM8K at 96.5%.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingagentictool-use
User reports sustained 80-90 t/s decode at full 262K context.
Setup uses MTP speculative decoding and prefix caching, with a 96.8% prefix cache hit rate and mean TTFT of 6.5s.
Multi-session throughput reaches about 3200 tokens/min across concurrent chats, and the benchmark ceiling is 106.9 t/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagentic
User reports DFlash2 speculative decoding with LMCache KV offload, with decode speeds varying by task: code 268 t/s, agent 165 t/s, mixed 97 t/s, prose 76 t/s.
Prefill runs at 15-17k t/s, and the KV cache pool holds 18M tokens on NVMe L2.
The setup required patches to avoid corruption.
User reports running GSM8K, MATH-500, HumanEval and MBPP locally.
User found reasoning stalls causing significant accuracy drops on hard math.
User notes the HumanEval score is possibly inflated due to contamination, and that harness bugs were fixed.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User compares llama.cpp built with -DGGML_CUDA_NCCL=OFF against the same build with it ON on 2x RTX 3090.
Setup is the unsloth/Qwen3.8-27B-GGUF UD-Q6_K_XL model at 229k context with an F16 KV cache, tensor split 24,24, and speculative decoding with draft-mtp.
NCCL ON degrades performance: PP 1046.92 vs 958.58 t/s and TG 59.67 vs 54.80 t/s.