Qwen3.6 reports

generation:

20.7 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B at 20.7 t/s at 127k context on an Intel Arc Pro B70 32GB, down from 70.7 t/s at depth 0. Setup is llama.cpp b10235 (Vulkan) with Q4_K_M, -ngl 999 -ub 512 -fa 0, single card in a PCIe-passthrough VM with 6 vCPU and 16 GB RAM. The user attributes the depth collapse to a memory-latency-bound decode attention kernel, measuring a constant ~21-25 ns per KV position per full-attention layer across backends, models and quants; vLLM XPU on the same card stays flat to 127k. Also reports a silent CPU fallback for prefill score tensors, a SYCL ~16.5 GiB weight-buffer limit, and a ~19% MoE decode regression from b8183 to b10235.

generation:

69.0 tokens/s

quant:

UD-IQ4_XS (GGUF)

kv:

q8_0

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B at 4-bit (UD-IQ4_XS) scoring 89.63% pass@1 on HumanEval at 69 tok/s on a single RTX 2080 Ti 22GB. Setup is llama.cpp with UD-IQ4_XS dynamic 4-bit (4.25 bpw), q8_0 KV cache, 16384 context, flash-attn on, whole model resident in VRAM with no offload. A routing patch (MoE expansion, 20 experts instead of 8 on layers 25-39) reached 90.85% pass@1 at 56 tok/s, a 19% decode speed cost; the user calls the +2 problems within statistical noise and notes the tests are original HumanEval, not EvalPlus.

generation:

95.5 tokens/s

prompt processing (prefill):

465.8 tokens/s

quant:

Q4_K_P (GGUF)

kv:

Q8_0

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B at 95.5 t/s decode and 465.8 t/s prefill on a mixed three-GPU Turing setup of 2x CMP 50HX 10GB plus 1x RTX 2080 Ti 22GB. Setup is llama.cpp with a ported DP2A patch (PR #25834) and -fmad=false, Q4_K_P GGUF weights, Q8_0 KV cache, Flash Attention, 262144 context, MTP speculative decoding at --spec-draft-n-max 3, tensor split 1,1,2.5 with the RTX as tail stage, ubatch 448. Baseline DP4A gave 47.8 t/s decode and 372.7 t/s prefill; DP2A alone 54.0 t/s; adding -fmad=false 62.2 t/s. A repo-28k workload measured 405.4 t/s prefill and 82.3 t/s decode. An end-to-end rerun showed 91.4 t/s raw eval, 4% below the 95.5 reference. Context 368640 also works but quality beyond the trained window was not measured.

generation:

101.3 tokens/s

prompt processing (prefill):

1841.7 tokens/s

quant:

AWQ (AWQ)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.6-27B-AWQ at 101.3 tok/s decode and 1841.7 tok/s prefill on dual modified RTX 2080 Ti 22GB cards with NVLink. Setup is vLLM 0.21.0 with AWQ Marlin, TP=2, MTP K=3, FlashInfer/FA2 attention, and FlashQLA SM70/SM75 legacy GDN prefill. The speed columns use the PP4096/TG128 repeat. The same rig reached a 735,084 token KV cache with turboquant_4bit_nc at max_model_len=262144, and passed a PP262000/TG1 gate at 785.26 tok/s prefill. A sequential 60-request Ragent6 run averaged 700.9 tok/s prefill and 35.2 tok/s generation. Gemma4 31B GPTQ reached 99.64 tok/s decode and 1655.65 tok/s prefill on the same runtime.

generation:

24.9 tokens/s

prompt processing (prefill):

611.0 tokens/s

quant:

Q5_K_M (GGUF)

kv:

q8_0

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User benchmarks Qwen3.6 27B Q5_K_M on dual AMD Radeon AI PRO R9700 GPUs, reporting 24.85 t/s generation and 611.02 t/s prompt processing at 32768 context on a single GPU. Setup is llama.cpp with ROCm, Q5_K_M GGUF weights, q8_0 KV cache, flash attention, and 2 GPUs with layer split. Dual GPUs roughly doubled long-context prefill (1215.87 t/s at 32768) but did not improve single-stream generation (24.31 t/s). The user also tested Gemma 4 31B, Qwen3.6 35B A3B, and Gemma 4 26B A4B, finding sparse-active models much faster but not directly comparable to dense models.

generation:

35-38 tokens/s

quant:

UD-Q6_K_XL (GGUF)

kv:

Q8_0

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports running Qwen3.6-35B-A3B UD-Q6_K_XL with a 256K context on an 8 GB NVIDIA GPU, keeping MoE experts in system RAM, at roughly 35-38 tok/s generation and 875-935 tok/s cold prefill. Setup is llama.cpp with Q8_0 K/V cache, CPU-MoE offload without mmap, on an Intel Core Ultra 7 265HX with 64 GB RAM; vanilla decoding, no speculative decoding. Longer cold-prefill testing stayed around 846 tok/s at roughly 58K processed tokens. MTP with Q4 and Q8 draft models was tested but varied with draft acceptance (one Q4 MTP-3 run reached only 52.8% acceptance and was slower than vanilla); DFlash was tested and gave only 15-17 tok/s. The setup is optimized for a single active inference slot.

generation:

89.5 tokens/s

prompt processing (prefill):

2665.3 tokens/s

quant:

5.4bit (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User benchmarks Qwen3.6-35B-A3B at 89.49 t/s generation and 2665.31 t/s prompt processing on a Mac Studio M3 Ultra. Setup is MLX with a 5.4-bit mixed-precision quant (4-bit baseline with important layers at 8-bit and BF16), 1024 prompt tokens and 512 generation tokens, peak memory 24.741 GB. The model is compared against 4-bit, 4.4-bit-msq, and 4.8-bit versions; the 5.4-bit quant achieves the lowest KL divergence (0.0189) but lower generation speed than the 4-bit baseline (108.50 t/s).

generation:

58.6 tokens/s

prompt processing (prefill):

671.4 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User benchmarks Qwen3.6-35B-A3B at 58.56 t/s generation and 671.40 t/s prefill on an M2 Max 32GB. Setup is llama.cpp build c1d0e7a00 with Metal backend, Q4_K_M GGUF (20.22 GiB), 4096-token prompt, no speculative decoding. The dataset reports mostly negative results: MoE beats dense Qwen3.8-27B by 5.4x generation, aggressive quantization bought little, MTP and DFlash2 speculative decoding were net losses on Metal, and q8_0 KV cache was free. MLX reached 86.9 t/s generation but only 296 t/s prefill versus llama.cpp's 57.8 t/s and 685 t/s. oMLX with paged SSD KV cache reached 13.7 t/s generation and 84 t/s prefill on the dense 27B.

generation:

5-8 tokens/s

quant:

GGUF (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports running Qwen3.6 35B A3B with MTP on a gaming laptop with 8GB VRAM, averaging about 5-8 t/s for the first 50k of context and dipping to 3-5 t/s for 50-100k before auto-compacting. Setup is llama.cpp with a GGUF quant and a custom web UI built on top of it; the model is partially offloaded to system RAM. The user says the speed is acceptable and they do other things while it runs, describing it as a fun local LLM project.

generation:

52-65 tokens/s

quant:

Q4_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B at 52-65 tok/s on an RTX 4060 Ti 8GB with 64 GB system RAM. Setup is llama.cpp with Q4_K_XL at 131k context, experts offloaded to system RAM and the rest in VRAM, on headless Linux. The user compares download defaults (~25 tok/s), tuned Windows (39-45 tok/s), and tuned headless Linux (52-65 tok/s). They also report Qwen3.8-Flash-Next 125B at 17-19 tok/s and Ternary Bonsai 27B at 36 tok/s.

prompt processing (prefill):

638-867 tokens/s

quant:

IQ2_XXS (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports a 35B-A3B MoE model (3B active) at 47-50 t/s decode and 160-200K context on a single Intel Arc B580 12 GB. Setup is llama.cpp SYCL with IQ2_XXS mixed quant, MoE experts offloaded to CPU via --n-cpu-moe 24, -ub 1024 -b 2048, -fa on, on Windows 11 with a Ryzen 5 9600X and 32 GB DDR5-6000. Decode drops to 43.0 t/s at 43.5K context; prefill at 43.5K is 638-867 t/s depending on quant and n-cpu-moe. A 65K context sweet spot with --n-cpu-moe 8 reaches 63 t/s. The same recipe applies across the Gated-DeltaNet MoE family, including Qwen3.6-35B-A3B, Occamy-1.0, Qwen3.8-35B-Distill and Ornith-1.5-35B-A3B.

generation:

66.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

kv:

q8_0

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B at 66.04 tokens/s on an AMD Radeon RX 9060 XT 16 GB with 32 GB of system RAM. Setup is llama.cpp (Vulkan backend) with UD-Q4_K_XL GGUF weights, q8_0 KV cache, 40k context, and MTP speculative decoding drafting up to 3 tokens per step. The 35B MoE model does not fit in 16 GB of VRAM, so the expert weights of the first 20 layers run on the CPU. Run on a Ryzen 7 5800X3D desktop under SteamOS, single slot, with flash attention and prefix cache reuse enabled.

prompt processing (prefill):

13816.2 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports a prompt-processing regression on Qwen3.6-35B-A3B, dropping from 13816.18 tokens per second to 6858.24 tokens per second between llama.cpp builds 4ebdf2c74 and bed0a8566. Setup is llama.cpp with Q4_K_M GGUF, 32768 context, -ngl 99 across 3x RTX Pro 6000 Blackwell (96 GB each) on a Threadripper PRO 9975WX with 256 GB RAM. The regression appears in llama-completion and llama-server but not in llama-bench, where both builds measure about 15600 t/s at pp18606.

generation:

2.5 tokens/s

quant:

int4 gs=64

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B at 2.45 tok/s on 4x Tesla M10 (Maxwell sm_50, 8 GB each, 32 GB combined). Setup is colibri with int4 gs=64 and 256-token context; all 10,240 experts held in VRAM (100% hit, zero CPU misses), while the dense DeltaNet/attention trunk runs on a Sandy Bridge CPU with no AVX2/FMA. The bottleneck is the CPU-side dense trunk, not the GPUs: a second CPU raises decode from 2.01 to 2.45 tok/s, and llama.cpp reaches 7.3 tok/s on the same hardware by offloading the trunk. A 1x GPU warm run gives 1.81 tok/s and a CPU-only run 0.09 tok/s.

generation:

98.8 tokens/s

prompt processing (prefill):

2358.0 tokens/s

quant:

4-bit

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User benchmarks Qwen3.6-35B-A3B at 98.8 tok/s decode and 2,358 tok/s prefill on an M3 Max 128GB using Splash 1.0.1 with 4-bit weights and DFlash speculative decoding. Setup is Splash 1.0.1 on macOS 26.6.2, 4-bit proprietary format, ~22K context, everything resident on GPU. Warm TTFT at 22K is 98 ms; 4 parallel streams reach 181 tok/s aggregate. User compares against Ollama NVFP4 (MLX) at 84.7 tok/s decode and oMLX serving Ornith 1.5 at 96.2 tok/s, and notes DFlash gains 2.74x on predictable text (239 tok/s) versus 87.2 tok/s on technical prose. User also benchmarks Qwen3.8-27B dense at 30.0 tok/s decode and 292 tok/s prefill, below Ollama's MLX engine at 42.5 tok/s.

generation:

14.4 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.6-27B-A3B-Coder at 14.4 t/s decode on an Intel Arc A770 16GB, scoring 10/10 on a Lua CSV parser acceptance task. Setup is llama.cpp with the SYCL backend and GGUF Q4_K_M weights, a comparison point against arcint's own AWQ IR serving on the same card. The figure is described as a rough bound rather than a directly comparable measurement, since the quantisation and engine differ from arcint's production configuration.

generation:

10.0 tokens/s

quant:

INT4 (OpenVINO IR)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.6 35B-A3B MoE at ~10 tok/s on an Intel Arc integrated GPU (Arrow Lake-P, Core Ultra 9 285H). Setup is OpenVINO 2026.2.0 with INT4/INT8 weights (~21GB) in OpenVINO IR format. The same MoE model runs at 17-19 tok/s under llama.cpp Vulkan on the same iGPU, and the user notes the MoE is slower than a dense Qwen3-8B (12.1 tok/s) on OpenVINO. Per-layer IR analysis shows 78 ops/layer x 40 layers = 3,120 unfused operations for the MoE MLP, causing GPU underutilization.

generation:

54.7 tokens/s

prompt processing (prefill):

615.0 tokens/s

quant:

UD-Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User benchmarks Qwen 3.6-35B-A3B at 54.7 t/s generation and 615 t/s prompt processing on a single Intel Arc Pro B70. Setup is llama.cpp with the SYCL backend, F16 accumulation, and UD-Q4_K_M quant (20.6 GiB). The post is a benchmark table from a GitHub repository; the flagship row is the single-card recommendation. The same table also lists Qwen 3.5-35B-A3B at 54.5 t/s, Gemma 4 26B-A4B at 52.6 t/s, and Qwen 3.5-27B at 20.4 t/s.

generation:

23.7 tokens/s

quant:

IQ4_XS (GGUF)

kv:

turbo3

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen 3.6 27B at 23.66 tok/s decode on a Tesla P40 24GB. Setup is llama.cpp with IQ4_XS GGUF, turbo3 KV cache, MTP self-speculative decoding (N=2, p_min=0.5), 65536 context, single card. MTP accept rate 83.7%, a +28.0% gain over the non-MTP Q4_K_M baseline at 18.48 tok/s. A cache sweep found turbo3 best at 22.43 tok/s and 18.1 GB VRAM, while q8_0 broke MTP with 0% draft acceptance.

generation:

9.7 tokens/s

quant:

int4 gs=64

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B at 9.67 tok/s decode on a single Tesla P40 24GB, with a cold run at 9.63 tok/s. Setup is colibri's CUDA backend built for sm_61, int4 gs=64 weights, 200 tokens greedy, single stream, whole model resident in VRAM (10240/10240 experts, 100% hit, 0 CPU misses). The P40 is slower than the P100 once warm (9.67 vs 12.02 tok/s) on ~346 GB/s GDDR5 versus ~732 GB/s HBM2; the cold-to-warm gap is 0.4% versus the P100's 61%. Probe-off control gives 9.62 tok/s and byte-identical output.