Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codinglong-context
User reports Qwen3.8 27B at 23.03 t/s generation and 2248.6 t/s prompt processing on a single RTX 4090 with 32 GB RAM.
Setup is llama.cpp via llama-swap with the unsloth UD Q4_K_XL GGUF, a Q8 KV cache, 131072 context, and a BF16 mmproj for multimodal input; cache reuse is disabled because of the multimodal model.
The user ran a three-day coding session in VS Code Copilot and hit a failure where the model emitted never-ending slashes; they ask what could cause it.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
agenticcoding
User reports generation at 12.0-13.1 t/s and prompt processing at 212-224 t/s for 8k-32k tokens.
Setup uses --n-cpu-moe 39 to offload experts to CPU, and requires 128 GB RAM.
Agentic tasks take 15-26 minutes. No quality benchmarks were run.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports DSpark draft model performance of 1-2 t/s with DeepSeek-V4-Flash on llama-server, compared to 30-40 t/s with the MTP draft model.
Setup is an RTX 4090 and an RTX 6000 Pro with 120 GB total VRAM and 32 GB RAM, running the unsloth DeepSeek-V4-Flash-0731 137 GB Q4 UD-IQ4-NL quant with the dspark-DeepSeek-V4-Flash-0731-Q8_0.gguf draft model, --spec-type draft-dspark, --spec-draft-n-max 3, --device-draft CUDA1, --tensor-split 2,3, --n-cpu-moe 13, --main-gpu 1, --split-mode layer, --ctx-size 65536, --flash-attn on, --batch-size 2048 and --ubatch-size 2048.
User asks for advice on the correct setup.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash 0731 at about 30 t/s prompt processing and about 3 t/s generation on CPU with an RTX 4090 and a Tesla P40.
Setup is the IQ4_XS quant at roughly 127 GB with MTP enabled, and layer-by-layer assignment because tensor splitting is not supported.
The user mentions an Unsloth 4bit K_XL quant at roughly 144 GB but could not run MTP with it due to memory constraints.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User compares Q5 and Q6 quants of Qwen3.8 27B on a three.js 3D arena game. Q6 took 3h32m at 6.01 t/s with 84,183 tokens, while Q5 took 13m23s at 71.98 t/s with 57,803 tokens.
Qwen3.6 Q5 was also tested at 84.79 t/s, and Q5 with medium reasoning reached 95.87 t/s.
The user notes Q6 ran at temp 0.6 instead of 1.0.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash (v4-flash-0731) at about 8 t/s on a single RTX 4090 with 64 GB RAM.
Setup is the Blaze engine with Unsloth's IXQ2/Q3 checkpoint, keeping heavily utilized experts in RAM with the CPU.
Throughput drops to 5 t/s on an expert miss that causes a disk read. Prefill is a work in progress.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports 194,048 context tokens at ~1650 prompt t/s and ~40 t/s generation without MTP, and 131,584 context tokens at ~1650 prompt t/s and ~65 t/s generation with MTP, up to 80 t/s for code, on an RTX 4090 FE with 48 GB DDR4 in WSL2.
Setup uses a q8_0 KV cache and q4_0 draft KV cache, with --cache-ram 32768, --ctx-checkpoints 32, --no-mmap and --mlock.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B on an RTX 4090, with throughput ranging from 44 t/s to 134 t/s and no RAM spilling.
Setup includes an mmproj file for vision.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User compares an RTX 4090 and an RTX 5090 for speed factors.
User mentions Muse Glimmer as an alternative but has not benchmarked it.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports 90 t/s with MTP enabled and 45-50 t/s without MTP at full context.
Setup uses a Q8 KV cache.
User sees no difference versus FP16 and prefers speed over context.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports prefill speeds of ~5000 pp/tg for DeepSeek V4 Flash on 4x RTX 4090 with a vLLM fork.
User also mentions Qwen3.8 Flash next with ~7500pp/135tg (MTP).
User compares to llama.cpp and ik_llama, noting vLLM is much faster for prefill.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
agentic
User reports DeepSeek V4 Flash at 10.9 t/s generation and 132.5 t/s prompt on an RTX 4090 with 128 GB of DDR5 5600 MT/s.
Setup is llama.cpp llama-server with unsloth's UD-Q2_K_XL quant, 64000 context, flash attention off and 8 threads pinned to the P-cores of an Intel Core Ultra 7 270k, on a 24GB GPU plus system RAM.
Pinning the P-cores took generation from 6.8 tok/s to 11 tok/s, and the IQ4_NL quant ran at 50.7 t/s prompt and 8.1 t/s generation with only about 10k context. The user finds it smarter than Qwen 3.6 27B Q4_K_XL but still prefers Qwen 3.6 27B for agentic work because it runs so much faster.
User reports running Qwen3.8-27b Q4 on an RTX 4090 to vibecode a Minecraft clone, with the model handling coding, audio, textures and 3D models.
User is impressed with the capability and the cost, which came in under $1.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User benchmarks Gemma 4 26B-A4B against Gemma 4 12B on an RTX 4090. The 26B-A4B runs at 138 tok/s using 15 GB of VRAM, while the 12B runs at 80 tok/s using 9 GB.
The 26B-A4B wins every scene and runs about 1.7x faster.
The user notes the 12B is ideal for a 16 GB laptop.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports MTP draft acceptance of about 73%.
Setup is a TBQ4_0 KV cache with an MTP draft of 3, using a fork of llama.cpp.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
text-generation
User reports about 150 t/s generation.
The model is described as a star performer.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports 3000 tokens in about 2 minutes, or 25 t/s, and is seeking faster performance.
Setup is Q4_0 with 120k context and both caches quantized to 4_0.
A reply adds a vLLM benchmark on an RTX 3090: a 27B INT4 quant at 125K context with a TurboQuant 3-bit NC KV cache and MTP speculative decoding, reaching 82 tok/s generation and 0.3-0.6s TTFT.