289 reports. These are individual setups, not a controlled benchmark. Qwen3.6 speeds by hardware →
2× NVIDIA RTX 3090 · 128,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User recommends getting enough GPUs to avoid VRAM hacks, and runs 2x RTX 3090s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.6-27B at ~70 t/s on dual RTX 3090s, generating interactive HTML content inline with chat.
2× NVIDIA RTX 3060 12GB · llama.cpp · 64,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports 43.26 t/s generation and 456 t/s prefill at 12k context on dual RTX 3060 cards. Setup is tensor parallel with MTP enabled and 64k context. Without MTP at 96k context, generation is 31 t/s. User praises the value and stability of CUDA.
User benchmarks Qwen3.6 27B BF16 and Qwen3.6 35B BF16. For the 35B, best generation throughput is 3500 t/s at 128 concurrency with MTP off, and prompt throughput is 30000 t/s. The 27B was also tested with MTP on and off.
2× AMD RX 9070 · llama.cpp · 131,072 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports generation around 45-52 t/s on two RX 9070 XTs with ROCm. Setup is MTP with spec-type = draft-mtp and spec-draft-n-max = 2, using the unsloth UD-Q5_K_XL GGUF. Prompt t/s varies, and draft acceptance rate is around 0.8-0.99. User praises the speed, smarts and steerability for agentic coding tasks.
NVIDIA RTX 4070 Super · ik_llama.cpp · 131,072 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User benchmarks llama.cpp at 89.76 t/s against ik_llama.cpp at 110.24 t/s on Qwen3.6-35B-A3B with MTP. Setup is the IQ4_XS quant on a Ryzen 7 9700X running CachyOS, with the GPU used as a secondary card and the iGPU for display. The ik_llama.cpp run is a 23% speed increase.
NVIDIA RTX 5080 · llama.cpp · 131,072 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports a 35B Q4_K_XL model at 128k context runs best with no MTP and --fit-target 1536. MTP does not help at 128k context. A 27B IQ3 model fits fully on GPU and benefits from MTP at 73 tok/s.
M2 Max 96GB · llama.cpp · 256,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User benchmarks Qwen 3.6 27B F16 on an M2 Max 96GB with llama.cpp and MTP speculative decoding, reporting 8-18 tok/s depending on the task. Without MTP the same setup reaches 6.6 tok/s. A Q8 quant was also tested but performed worse. Context up to 150k+ tokens is usable. The user notes chat template fixes were critical, and the setup was used for agentic coding to create a Pacman game.
NVIDIA RTX 3090 · BeeLlama · 128,000 ctx
User benchmarks KV cache quantization methods with Qwen3.6 27B at 64k and 128k context. The IQ4_XS quant is also tested.
NVIDIA RTX 3090 · ik_llama.cpp · 156,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6-27B-MTP-IQ4_KS.gguf at 72.9 t/s decode and 1261 tok/s prefill on an RTX 3090 24GB at 156k context. Setup is ik_llama.cpp with a q8_0/q8_0 KV cache, MTP, and vision on CPU. The user also tested llama.cpp and BeeLlama.
2× AMD MI50 32GB · llama.cpp
User benchmarks MTP KV cache quantization at Q8_0 against no quantization on Qwen3.6-27B-Q8_0 with llama.cpp. Setup uses tensor parallelism on 2xMI50 32GB. Wall time difference is negligible at roughly 0.14s with the quantized draft KV cache. Aggregate accept rate is 0.735-0.741.
AMD Strix Halo 128GB · llama.cpp
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports MTP enabled with --spec-type draft-mtp --spec-draft-n-max 3. Baseline without MTP is 11.7 tok/s. Q8_0 was also tested: 7.4 → 18.1 tok/s (2.44×).
AMD Strix Halo 128GB · llama.cpp
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen 3.6-27B Dense with MTP on Strix Halo under Windows, at 18.7 t/s for a poem, 19.8 t/s for editing HTML and 23.2 t/s for creating HTML. Setup is llama.cpp with spec-draft-n-max 3.
AMD Strix Halo 128GB · llama.cpp
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen 3.6-27B Dense with MTP at 14.5 t/s on a poem task, 14.2 t/s on an edit HTML task, and 17.9 t/s on a create HTML task, on Strix Halo. Setup is llama.cpp on Windows with spec-draft-n-max 6.
AMD Strix Halo 128GB · llama.cpp
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen 3.6-27B Dense without MTP at 12.5 t/s on a poem task, 12.6 t/s on an edit HTML task, and 12.6 t/s on a create HTML task on Strix Halo. Setup is llama.cpp on Windows.
AMD Strix Halo 128GB · llama.cpp · 128,000 ctx
User benchmarks MTP against non-MTP for 27B and 35B-A3B models. The 27B-MTP run shows significant speedup in generation and overall wall time for long-context chat. The 35B-MTP run shows mixed results: faster generation but slower end-to-end due to prefill overhead.
AMD Strix Halo 128GB · llama.cpp
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports multiple models and quants on Strix Halo 128GB with the ROCm backend, plus tests on an RTX 3090 and an RTX 5070. Setup is the Q4_K_M quant for a chat workload.
4× NVIDIA RTX 3090 · vLLM
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks Qwen3.6-27B FP16 on 4x RTX 3090 with vLLM at TP=4. Setup is a power limit sweep from 200W to unrestricted, with peak efficiency at 220W. User reports Qwen 3.6 27B works well as a daily driver.
NVIDIA RTX 3060 12GB · llama.cpp · 32,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports a dense model at 11.9 GB GGUF size, barely offloading to CPU. The user wants to try a higher quant, having chosen this one because of the 11.9 GB GGUF size.
User reports inconsistent output from local models Gemma 4 and Qwen 3.6 35B A3B compared to frontier models, for a dictation and SOAP scribe. Setup is a Core Ultra 9 with 128 GB RAM and an RTX 5090, running Proxmox with AnythingLLM and Ollama on llama.cpp. The system prompt is a 25-30k token markdown file.