Qwen3.6 reports

generation:

27.6 tokens/s

quant:

Q6_K (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.6-27B-Q6_K at 27.55 t/s writing SVG images, on hardware the source does not name. Setup is temperature 0.6, top_p 0.95, top_k 20, min_p 0.0, with open-visual in Open WebUI to show the images. Across six SVG prompts the runs take 3min 10s to 8min 24s at 27.05 to 27.55 t/s.

generation:

105-108 tokens/s

quant:

INT4 (safetensors)

kv:

Q8

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.6-27B-INT4 at 105-108 t/s generation on an RTX 5090 at 256k context. Setup is vLLM 0.19 with the Lorbus AutoRound quant, an fp8_e4m3 KV cache, flashinfer attention, and MTP speculative decoding with 3 tokens.

generation:

50-60 tokens/s

quant:

IQ4_XS (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User switched from Qwen3.6 35b-a3b IQ4_XS to Qwen3.6 27b IQ3_M. The 35b-a3b ran at 50-60 t/s with slow prompt processing, while the 27b ran at ~40 t/s with consistent performance. User reports the 27b found a bug the 35b could not, and notes the dense model handles compression better than the MoE.

generation:

30.0 tokens/s

quant:

Q5_K_S (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B at about 30 t/s with Q5_K_S at 128k context on an RTX 3070 8GB with 64GB DDR4. Setup is llama.cpp. Q4_K_XL reached 32 t/s, and IQ4_XS gave 25-30 t/s at 32k context. The user found that larger quants (Q4_K_XL, Q5_K_S) gave better speed than the smaller Q4 (IQ4_XS).

generation:

80.0 tokens/s

quant:

NVFP4 (safetensors)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.6-27B at ~80 t/s on an RTX 5090 at 218k context. Setup is vLLM 0.19.1rc1 with NVFP4 quantization on a single card.

generation:

19.2 tokens/s

prompt processing (prefill):

186.8 tokens/s

quant:

Q4_K_M (GGUF)

kv:

Q8

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports running Qwen3.6-27B Q4_K_M on an RTX 5070 Ti 16GB and an RTX 2060 6GB with llama-server, reaching pp 186.76 t/s and tg 19.21 t/s at 71k actual context. Setup is llama-server with the Q4_K_M quant across the two cards. llama-bench results with CUDA show tg speeds around 16-25 t/s depending on configuration.

generation:

22.5 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User benchmarks Qwen 3.6 27B across BF16, Q4_K_M, and Q8_0 GGUF quant variants with llama-cpp-python using Neo AI Engineer, on HumanEval, HellaSwag, and BFCL. Q4_K_M is the best practical variant: 1.45x faster than BF16, 48% less peak RAM, and a 68.8% smaller model file. Function calling scores are nearly identical to BF16.

prompt processing (prefill):

913.0 tokens/s

quant:

Q4_K_M (GGUF)

kv:

Q4

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports speculative decoding with DFlash on a single RTX 3090, targeting Qwen3.6-27B Q4_K_M GGUF at ~16 GB with a z-lab Qwen3.6-27B-DFlash bf16 draft at ~3.46 GB. Setup is DDTree tree-verify with block size 16, budget 22, and greedy verify. The KV cache is compressed to TQ3_0 at 3.5 bpv, roughly 9.7x versus F16, with a 4096-slot ring buffer enabling 256K context in 24 GB. Decode uses sliding-window flash attention with a 2048-token window, and prefill ubatch auto-bumps from 16 to 192 for prompts over 2048 tokens. The endpoint is OpenAI-compatible and CUDA only, with no Metal, ROCm, or multi-GPU support. Output is bit-identical to autoregressive in AR mode, and the draft matches the z-lab PyTorch reference at cos sim 0.999812.

quant:

IQ4_XS (GGUF)

kv:

Q8

rating:

5/5

User reports Qwen 3.6 27B is excellent for pyspark/python and data transformation debugging, running on an ASUS ROG Strix SCAR 18 with an RTX 5090 laptop (24 GB VRAM) and 64 GB DDR5 RAM. Setup is llama.cpp with the IQ4_XS quant at 200k context and a Q8_0 KV cache. The user initially tried q4_k_m at q4_0. No tokens/sec is reported. The user is cancelling cloud subscriptions due to local performance.