Qwen3.6 reports

generation:

23.0 tokens/s

prompt processing (prefill):

600.0 tokens/s

quant:

Q4_K_M (GGUF)

kv:

Q8_0

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive at ~600 tok/s prefill and 23 tok/s decode on an RTX 2060 6GB with 32GB DDR4 RAM. Setup is llama.cpp with Q4_K_M and Q8_0 KV cache at 131072 tokens of context, with 39 MoE expert layers on CPU and the vision projector on CPU to keep VRAM under ~5.2 GB. At around 90k context the speeds settle to roughly 485 tok/s prefill and 15 tok/s decode.

generation:

30.0 tokens/s

quant:

Q4_K_XL (GGUF)

kv:

Q8_0

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports 30 t/s generation on an RX 6700 XT 12GB with a Qwen3.6 35B-A3B Tiel-Coder model at Q4_K_XL, using llama.cpp with the Vulkan backend on Windows 11. Setup uses 100,000 context, Q8_0 KV cache, flash attention, 28 MoE layers offloaded to CPU, 6 threads, and MTP speculative decoding with draft-n-max 3. User is asking for advice to reach 40 t/s without changing the quant, and notes 6 threads was best out of 4, 8, and 12.

generation:

80.0 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B at close to 80 tok/s on a Strix Halo mini PC with 64GB of memory. No engine, quant or context length is stated. The user also notes Qwen 27B models run at about 30 tok/s on the same machine. The user asks whether Flash-Next will run on this hardware.

generation:

53.2 tokens/s

prompt processing (prefill):

891.6 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.6 35B-A3B at 53.2 t/s generation and 891.6 t/s prefill on a 29,982-token prompt on an AMD Strix Halo 128GB. Setup is llama.cpp at Q4_K_M, one request, speculative decoding off. AGmind Systems Lab's doc-session run of 2026-08-03 (CC BY 4.0). A source-reported measurement, not independently reproduced.

generation:

24.1 tokens/s

prompt processing (prefill):

322.7 tokens/s

quant:

UD-Q8_K_XL (GGUF)

kv:

q8_0

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B at around 25 tok/s on a Strix Halo 128GB box (Minisforum MS-S1 MAX, Ryzen AI Max+ 395). Setup is llama.cpp b8890 with UD-Q8_K_XL GGUF and q8_0 KV cache, Vulkan RADV backend, 524288 context, 3 parallel slots, running under 100W. Live llama.cpp metrics at capture: 153562 tokens max context, 322.685 prompt tokens/s, 24.0673 predicted tokens/s. User notes long context needs architecture, quant, KV cache and backend to line up, and plans to test UD-Q3_K_XL next.

generation:

120-140 tokens/s

quant:

UD_Q4_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.6 35B-A3B at 120-140 t/s generation on two Tesla P100s, with prefill around 1000 t/s at 0 context and 700 t/s at 13k context. Setup uses Unsloth's UD_Q4_K_XL quant; the engine is not named. The user praises the model for broad agentic tasks and notes it is fast enough to be conversational, but criticizes it for coding, persona holding, and hallucination. They also mention running Qwen3.8 Flash-Next at 30-40 t/s with Strata at Q3.

generation:

30.0 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.6 35B-A3B at 30 t/s on an RTX 3050 6GB with 24GB system RAM, dropping to around 22 t/s once the 64K context is full. Setup uses the Unsloth Q4_K_M quant with 64K context; the model is partially offloaded to system RAM. User is building a voice agent with Moonshine Small and Faster-Whisper STT/TTS and asks for better voice model combinations and a faster alternative to Qwen3.6 MoE.

generation:

57-60 tokens/s

quant:

UD-Q2_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B at 57-60 tok/s on a rented V100 16GB. Setup is AgrillaMoE, a fork of llama.cpp's server, with the 2-bit UD-Q2_K_XL quant and a MoE-expansion patch raising active experts to 20 on layers 25-39. The user says the expansion patch improves GPQA-Diamond from 81.82% to 84.34% with Q8_0, and that the server speaks both OpenAI and Anthropic APIs.

generation:

81.8 tokens/s

prompt processing (prefill):

639.9 tokens/s

quant:

Q2

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B at 81.85 t/s generation and 639.85 t/s prefill on an AMD BC-250 16GB at 2K context. Setup is QuarkStar, a new Vulkan/Metal inference engine, with Q2 quantization and fully resident inference on 16 GB. User also reports M2 Pro 16GB results: 37.78 t/s generation and 448.75 t/s prefill at 2K, 31.08 t/s and 270.02 t/s at 8K, and 25.64 t/s and 177.21 t/s at 16K. The engine also supports KAT-Coder-V2.5-Dev and bounded SSD expert streaming for smaller machines.

generation:

24.2 tokens/s

prompt processing (prefill):

65.6 tokens/s

quant:

IQ4_XS (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B at 24.2 t/s generation and 65.62 t/s prompt eval on an RTX 4060 8GB with 32GB DDR5 RAM. Setup is llama.cpp with IQ4_XS quant and --moe-cache-mib 2048, using the PR#29887 MoE expert cache in host memory. User is not getting expected t/s and asks for help; also tried Qwen3.8-Flash-Next Q2 at 8.02 t/s generation and 6.96 t/s prompt eval.

generation:

23.0 tokens/s

quant:

Q4_K_M (GGUF)

kv:

q4_0

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports roughly 23 t/s for Qwen3.6-35B-A3B UD-Q4_K_M on an RTX 2060 6GB with default ik_llama.cpp, and around 26-30 t/s with their own fork's expert-residency changes. Setup is a single RTX 2060 6GB with a Ryzen 7 2700X and 40GB DDR4, Q4_K_M weights with all experts on the CPU (-cmoe), at around 15-30k context. The fork adds bandwidth-adaptive CPU/GPU execution and a resident expert cache in spare VRAM (--moe-resident auto --moe-resident-mib), giving the user roughly a 20-30% speedup.

generation:

7-11 tokens/s

kv:

turbo4

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.6 27B at 7-11 tok/s with stock llama.cpp on an NVIDIA DGX Spark (GB10), consistent across every scenario tested. Setup is stock llama.cpp on one GB10 with no speculative decoding and a q8_0 KV cache, the whole model in unified memory; the post does not state the target quant for the stock run. With DFlash speculative decoding (spiritbuun fork, Q4_K_M target, Q8_0 draft, turbo4 KV) the author gives per-task ranges only: 38-40 tok/s HTML/JS, 24-25 Python, 23-25 short chat, 20-22 medium context and 27-29 sustained over 2048 tokens. Q8_0 and BF16 targets are slower. vLLM with MTP reached around 30 tok/s. The 35B-A3B MoE runs 60-66 tok/s stock and up to 113 tok/s on HTML/JS with DFlash.

generation:

120.0 tokens/s

quant:

AWQ 4-bit (safetensors)

kv:

turboquant_k3v4_nc

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B at AWQ 4-bit decoding at about 120 tokens/second under a single request on 2x modded 22GB RTX 2080 Ti with NVLink. Setup is vLLM with tensor parallelism across the two cards. The post's main benchmark ran a W8A8-INT8 build with a TurboQuant KV cache (about 395K tokens) and up to 16 concurrent requests, but those results appear only in screenshots; the author notes AWQ 4-bit is the better choice on Turing.

generation:

154.1 tokens/s

quant:

UD-Q4_K_S (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B at 154.12 t/s with MTP on 2x AMD Radeon PRO W7800 48GB. Setup is llama.cpp (Lemonade stack) with UD-Q4_K_S GGUF and tensor-split across both cards; stock decode without MTP is 79.86 t/s, and MTP drops to 146.7 t/s at 8k and 132.6 t/s at 32k. Part of a registry PR adding 40 recipes for the W7800 48GB; other models in the table include Qwen3-Coder 30B-A3B, Ornith 1.5 35B-A3B, Gemma 4 26B-A4B-it, Qwen3.8 Uncensored and Qwen3.6-27B.

generation:

104.5 tokens/s

quant:

MLX 4-bit (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B at 104.5 tokens/s single-stream decode on an Apple Mac mini M4 Pro with 48 GB unified memory. Setup is Rapid-MLX 0.15.4 with MLX 4-bit weights (17.5 GB on disk). The figure is the median of three runs at temperature 0. Separately, the source reports 106.5 tokens/s aggregate with four concurrent streams and a 2.4 s TTFT for a 2K-token prompt. The 23.2 GB Metal peak covers the whole sweep, not single-stream use. These are developer-produced measurements; the linked raw-data directory returned a 404 when checked.

generation:

24.6 tokens/s

quant:

4-bit (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.6-27B 4-bit at 24.6 t/s with native MTP speculative decoding on an M4 Pro, up from 15.7 t/s baseline (1.57x). Setup is MLX with the MTP head enabled at temp=0, 3 runs x 8 prompts, acceptance 88.3% (A/V). At temp=0.6 MTP gives 24.0 t/s (1.53x) and at temp=1.0 23.5 t/s (1.50x). Qwen3.5-27B 4-bit goes 15.3 to 24.0 t/s (1.57x) and Qwen3.5-35B-A3B 4-bit only 85.3 to 87.9 t/s (1.04x), the MoE gain being marginal. Other users' results cited: M2 Ultra 8-bit 35B-A3B 1.11x and 122B-A10B 1.09x, M5 Max 4-bit 9B +11.3%, 27B +35.5%, 122B +12.4%, M5 Pro 4-bit 27B +31.4%.

generation:

10.4 tokens/s

prompt processing (prefill):

118.8 tokens/s

quant:

IQ4_XS (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports ik_llama.cpp at 118.79 t/s prompt processing and 10.40 t/s generation on a GTX 1660 Super 6GB with Qwen3.6-35B-A3B IQ4_XS at 131072 context. Setup is ik_llama.cpp with --cpu-moe, -ngl 999, -amb 2048, -rtr and --prompt-cache-all on an AMD Ryzen 7 5800X. Mainline llama.cpp on the same machine reached 239.81 t/s PP and 15.69 t/s TG, so ik_llama.cpp is about 2x slower on PP and 1.5x slower on TG. User suspects --prompt-cache-all, -rtr or -amb 2048 may be responsible.

generation:

111.7 tokens/s

prompt processing (prefill):

4326.8 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B at 111.72 tok/s decode and 4,326.76 tok/s prompt eval on an RTX 5070 Ti 16GB. Setup is llama.cpp with Q4_K_M GGUF, F16 KV cache, 65,536 context, cache 132 and ub=8192, single request. Peak VRAM was 14,369 MiB. At NP4 the same model reached 247.79 tok/s aggregate decode (62.00-67.78 per request) and 1,142.11-2,243.00 tok/s prefill per request. Gemma-4-26B-A4B-it Q4_0 runs are also listed, reaching 104.58 tok/s at NP1 and 175.56 tok/s aggregate at NP16.

generation:

71.3 tokens/s

quant:

NVFP4 (MLX)

kv:

q8_0

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B at 71.3 t/s decode on short prompts on a Mac Studio M2 Max 64GB. Setup is Ollama with NVFP4 MLX weights, q8_0 KV cache, 32768 context, flash attention on, single concurrent request, think disabled. Decode stays near-flat with prompt length (69.3 t/s medium, 67.8 t/s long) and the 5-turn multiturn total is 23.7 s; the model is the recommended default coding model, replacing Qwen3-Coder-30B.

generation:

110.0 tokens/s

quant:

UD-IQ3_XXS (GGUF)

kv:

turbo4

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B at ~110 t/s on an AMD RX 7800 XT 16GB with MTP and turbo4 KV cache at 32K context. Setup is a NJannasch llama.cpp mtp-turboquant fork on ROCm 6.4.0 (gfx1101) with UD-IQ3_XXS GGUF, turbo4 KV cache, --spec-type draft-mtp, and --reasoning-budget 512. MTP acceptance was 86% at 32K context; turbo4 KV saved ~470 MB vs f16 at 32K. The user warns that a silent CPU fallback can drop generation to 5-10 t/s and that gfx1032 cards crash with this fork.