289 reports. These are individual setups, not a controlled benchmark. Qwen3.6 speeds by hardware →
NVIDIA RTX 4090 · llama.cpp · 262,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports MTP draft acceptance of about 73%. Setup is a TBQ4_0 KV cache with an MTP draft of 3, using a fork of llama.cpp.
NVIDIA RTX 3060 12GB · llama.cpp · 32,768 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports llama-bench results of about 914 t/s for pp512 and about 46.8 t/s for tg128. A practical coding profile at 32k context generates at about 43.4 t/s. MTP gave about 47.7 t/s, a 2% improvement.
M2 Max 96GB · llama.cpp · 262,144 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports MTP speculative decoding gives a 2.5x speedup on an M2 Max 96GB. Setup is Q5_K_M quant with a q4_0 KV cache. The user also provides hardware recommendations for various Apple Silicon and NVIDIA GPUs.
NVIDIA RTX 5090 · vLLM · 200,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports generation speed varying 59-111 tok/s with MTP enabled and 3 speculative tokens. Setup uses an fp8_e4m3 KV cache, with prefix caching tested. Mean MTP acceptance length is 2.28. A stability pass at 200k context completed 10/10 runs.
NVIDIA RTX 5090 · llama.cpp
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports a 6% speed increase from MTP on a grafted model at Q4 on an RTX 5090. User also tests Q8 on an RTX 5090 plus an RTX 3090, reaching 148.20 t/s without MTP and 152.02 t/s with MTP.
NVIDIA RTX 3090 · llama.cpp · 100,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports MTP (Multi-Token Prediction) GGUF with the am17an commit for faster inference. Setup uses a Q4_0 KV cache and a speculative draft of 2. At 100k context, VRAM usage is about 19 GB, and performance degrades above 90k context.
NVIDIA V100 32GB · llama.cpp · 200,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports 29-30 t/s without MTP and 54-55 t/s with MTP on the MTP branch of llama.cpp by am17an, at a 150W power limit. The model is used as a VSCode Copilot backend. Throughput falls to 40-45 t/s after 50k tokens.
NVIDIA RTX 5060 Ti 16GB · llama.cpp · 75,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6 27B IQ4_XS at 22 t/s generation and 760 t/s prompt processing on an RTX 5060 Ti 16GB. Setup is llama.cpp with TheTom's TurboQuant fork, a 75k context window, and a turbo4/turbo2 KV cache. The user also tested BF16, Q8_0, Q6_K, Q5_K_XL, Q4_K_XL, IQ4_XS, IQ3_XXS, Q3_K_XL, Q3_K_M and Q2_K_XL on an L40S or RTX 5060 Ti, comparing quality with a chess board SVG generation task, and recommends IQ4_XS as the minimum.
M5 Max 64GB · MLX
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports 63 t/s on Qwen3.6-27B 4-bit MLX on an M5 Max 64GB with the MTPLX engine, up from a 28 t/s baseline. Setup is a custom patched MLX fork with Metal kernels, using native MTP heads at temperature 0.6, top_p 0.95 and top_k 20, with optimal depth D3.
RTX 5000 PRO 48GB · vLLM · 196,608 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports 60-90 t/s with a typical 80 t/s on vLLM 0.20.1 with CUDA 12.9. Setup uses a BF16 KV cache and MTP=2 speculative decoding.
User compares Qwen3.6-27B with and without thinking against Coder-Next. Qwen3.6-27B with thinking disabled is the most consistent at a 95.8% ship rate, and Qwen3.6-27B and Coder-Next are statistically tied overall. Qwen3.6-35B-A3B performed poorly and was dropped.
RTX 2060 Max-Q 6GB · llama.cpp · 65,536 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.6-35B-A3B, a 3B-active MoE, running on a 5-year-old laptop with an RTX 2060 Max-Q 6 GB and 24 GB RAM. Setup is llama.cpp with a Q8 KV cache, flash attention, and 64k context. A long context variant uses 128k context with Tom's fork.
AMD RX 7900 XTX · 32,768 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User is considering adding a second 7900 XTX for 48 GB VRAM to run larger models. Currently runs Qwen 27B Q6 dense with 32K context at 32 t/s prompt processing. Main use case is coding via opencode.
M5 Max 128GB · MLX · 290,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports 160 t/s prefill and 5-6 t/s generation on an M5 Max 128GB with Qwen 3.6 27B Q8 MLX at 290k context. GPU utilization sits at only 36-50%. The user finds the generation rate low against an expected 8-14 t/s and asks others for comparison.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen 3.6 27B at 32 t/s on a MacBook Pro M5 Max 64GB, generating 33,946 tokens in 18m04s. User compares Gemma 4 31B on the same hardware at 27 t/s, generating 6,209 tokens in 3m51s. User finds Qwen showed more creativity while Gemma won for game logic.
NVIDIA RTX 3090 · vLLM · 218,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports about 50 to 66 t/s at roughly 218K context on text, narrative and code workloads. Tool calls with outputs of about 25K tokens now complete without OOM after a Genesis patch fix (PN12). Throughput is lower than an earlier configuration, but context length and stability are higher.
M3 Max 128GB · MLX · 290,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports 160 t/s prefill and 5-6 t/s generation on an M5 Max 128GB with Qwen 3.6 27B Q8 MLX at 290k context. GPU utilization sits at only 36-50%. User expected 8-14 t/s generation and asks how other setups compare.
2× NVIDIA H100 80GB · vLLM · 128,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.6-27B at 45 t/s on a rented GPU instance with 2x H100s and 160 GB of VRAM. Setup is vLLM at 128K context, serving multiple agents hitting the API at once, including Claude Code, QwenCode and social media bots. The instance costs about $0.90/hr, with $120 spent last month. The user says the model outperformed a 120B model in tests.
NVIDIA RTX 4090 · LM Studio · 120,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports 3000 tokens in about 2 minutes, or 25 t/s, and is seeking faster performance. Setup is Q4_0 with 120k context and both caches quantized to 4_0. A reply adds a vLLM benchmark on an RTX 3090: a 27B INT4 quant at 125K context with a TurboQuant 3-bit NC KV cache and MTP speculative decoding, reaching 82 tok/s generation and 0.3-0.6s TTFT.
16GB VRAM (Vulkan backend, unspecified GPU model) · llama.cpp · 80,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6-35B-A3B at 37.96 ± 0.10 t/s generation and 1021.05 ± 1.24 t/s prompt processing on a 16 GB VRAM GPU. Setup is llama-server.exe with the Vulkan backend, the Unsloth IQ4_XS quant at ~4.25 bpw, --n-gpu-layers 99, --n-cpu-moe 16 to offload MoE experts to CPU, --threads 14, --batch-size 1024, --ubatch-size 1024, --flash-attn 1, q8_0 KV cache, --ctx-size 80000, --cache-ram 2048 and --no-mmap. The generation figure is tg1000 and the prompt processing figure is pp80000. The user runs it with a pi coding agent for file operations, tool calls, summarizations and MCP calls, and says it is usable on a 16 GB GPU.