Qwen3.6 35B (3B active)
on NVIDIA RTX 3060 12GB · llama.cpp · 32,768 ctx
Summary
User reports Qwen3.6-35B-A3B at a median 79 tok/s generation on a single RTX 3060 12GB with 32GB DDR5.
Setup is llama.cpp with Q4_K_M and q8_0 KV cache, using --cpu-moe with a 7500MiB MoE expert cache, mlock, and MTP speculative decoding at 32768 context. Prompt processing is around 800 tok/s under 1k tokens, ~1100 tok/s for 1k-4k, and ~1190 tok/s for 4k-16k. MTP draft acceptance is around 85% and VRAM sits at ~11.4GB.
The 79 tok/s median comes from 433 responses longer than 500 tokens, with most between 76 and 83 tok/s. A fixed 3.6k prompt / 700 token output test showed 50-58 tok/s with --n-cpu-moe 25, 71-73 tok/s with --cpu-moe plus cache, 75-81 tok/s with mlock, and 77-84 tok/s after updating llama.cpp. Forcing 4 experts reached 91-95 tok/s but caused looping on long outputs.