Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports 120.1 t/s decode on narrative text and 218.3 t/s on code with vLLM v0.26.1rc1 and AutoRound INT4 (group 128) on 2x RTX 3090 over PCIe Gen4 x16/x16 with no NVLink and patched P2P.
Prefill reaches 1342 tok/s at 10k and 628 tok/s at 90k. Speculative decoding uses 7 draft tokens with an acceptance length of 3.35 and 47.8% acceptance.
Peak VRAM is 22.3 GB per card with zero leak, and the context ceiling is 131k because the DFlash2 drafter takes about 13.5 GB. Power is capped at 220/250 W, and the user used Kimi K3 for vLLM fixes.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingsummarization
User reports a custom C + Metal runtime with speculative decoding at 18 t/s end-to-end on coding tasks and 17.7 t/s for an LRUCache implementation, on 36 GB unified memory.
Q4 weights are about 15 GB, mmap'd. Prose runs at roughly 10-11 t/s. TTFT is about 1.4s for a short prompt and about 2.7s for a 128-token prompt.
The user compares the custom runtime against llama.cpp and reports it faster.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User compares an RTX 4090 and an RTX 5090 for speed factors.
User mentions Muse Glimmer as an alternative but has not benchmarked it.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen 3.8 27B UD Q4_K_XL on an RTX 5070 Ti Mobile 12GB with llama.cpp, at roughly 1.5-5 t/s in a context-prioritized configuration and roughly 9-11.5 t/s in a speed-prioritized one.
Setup uses a q8_0 KV cache, with context at 262144 in the first configuration and 98304 in the second.
User praises the model's capability and instruction following.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B at about 20 t/s on 16 GB of VRAM at 130k context.
Setup is a PR that adds a --n-cpu-ffn option for dense models, with the Q4_K_M quant.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B at 55 tok/s on an RTX 3090.
Setup is the IQ4_XS quant.
The user also mentions GLM-5.3 pricing and intelligence, DeepSeek V4 Flash, and Strix Halo hardware, but does not run them.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports vectionlabs/Salience-27B-R5-GGUF at 55-60 t/s and used it to build a Tibia clone in HTML/CSS/JS.
Setup uses MTP and vision.
The title mentions Qwen 3.8 27B, but the setup names bartowski/vectionlabs_Salience-27B-R5-GGUF.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B at about 68 t/s generation on dual RTX 5060 Ti.
Setup is speculative decoding with a Q8 KV cache at 100k context. Vision is enabled but not used.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports a dense 27B model on a Strix Halo APU, with a DFlash2 drafter giving +40% decode over MTP.
Setup is Nathan's Vulkan fork of llama.cpp, with Q5_K_XL decoding faster than Q4_K_XL due to higher draft acceptance.
Prefill is ~300 t/s at 3k context. FP4 builds are slower and worse quality.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports llama.cpp benchmarks on a dual-GPU setup of an RTX 4090 laptop and an RX 7900 XTX over TB4, reaching full 262k context with +70% generation and +40% prefill.
Setup uses the CUDA backend for the NVIDIA card and Vulkan for the AMD card, with a q8_0 KV cache to enable 262k context at some cost in speed. MTP speculative decoding gave +50-70% generation but halved prefill, and the user filed a bug. A Q6_K_XL quant over Q8_0 gave +9% generation and more context on CUDA, and rebuilding to a newer llama.cpp gave +13% generation for free.
User reports Qwen3.8 27B Instruct scoring 36 and 34 composite on two ACT practice tests, with perfect reading scores.
Setup is 2x RTX 3090 with full offload and 32k context.
The two tests took 177 minutes in total.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
agenticcoding
User reports 50-65 t/s generation on dual RTX 3090, sometimes dropping to 40s.
Setup is llama.cpp with the Qwen3.8-27B-UD-Q6_K_XL GGUF, --mmproj for vision, --kv-unified, --flash-attn, and q8_0 KV cache types, plus speculative decoding with draft-mtp.
The use case is agentic coding.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
creative-writing
User reports Qwen3.8 27B generating a GTA-style game, with prompt processing of ~850-890 t/s for 25k tokens and generation typically 50-55 t/s, peaking at 197 t/s with ngram speculative decoding.
Setup is llama.cpp with ngram and MTP draft, a Q8 KV cache, and 128k context.
User is enthusiastic about the results.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports a DFlash2 draft model with a Q4_K_M quant, giving roughly 2x decode speedup with 5 draft tokens and roughly 2.7x with 7 draft tokens.
Setup is a 4-bit Unsloth model at 256k context, with draft acceptance identical to Q8_0.
Prefill slows by roughly 15% with the draft model.
User reports Qwen3.8-27B at 495 t/s on a single MI300X at 1M context, up from 311 t/s.
Setup is dstack's toolkit with source-level patches to SGLang's AITER attention backend.
The figure is for four concurrent users at 10k in / 1.5k out, with p50 TTFT under 1.5s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B at 382 t/s on an RTX 3090 when reproducing context, and about 133 t/s on real chat prompts.
Setup uses an fp8 KV cache, int8 activations, DFlash2 block drafting, lookup-augmented drafting, prefix caching, split-KV verify attention, and KVarN for 262k context.
Quality is unchanged, with GSM8K at 96.5%.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports 90 t/s with MTP enabled and 45-50 t/s without MTP at full context.
Setup uses a Q8 KV cache.
User sees no difference versus FP16 and prefers speed over context.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports the Escha 2-bit quant of Qwen3.8-27B, 10.15 GB on disk, averages about 100% of FP8 performance across 8 benchmarks.
On LiveCodeBench v6, Escha W2 scores 86.81 against 85.16 for FP8.
The previous 2-bit Qwen3.6-35B MoE scored 62.6 against 67.0 for FP8 on the same benchmark.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8 27B at ~23 t/s on 3x RTX A4000 16GB.
Setup is llama.cpp with Q8_0 quantization, MTP draft mode and ngram speculative decoding.
User asks for optimization advice.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
agenticlong-context
User reports Qwen 3.8 27B FP8 on two RTX 6000 Pro GPUs with sglang at 150 tk/sec, but tasks take 12x longer than Claude Opus 5.
BF16 on the primary card with FP8 offloading took 3 hours versus 1h45m for FP8. Context lengths tested are 256k, 128k (too small) and 500k (faster despite being unsupported).
A skill that reads large documentation files hits 80k context before starting. The user compares against Claude Code, Pi Code, Qwen Code and Hermes, and uses ChatGPT 5.6 high for testing.