Qwen3.6 35B (3B active)
on NVIDIA RTX 4060 Ti 16GB · llama.cpp · 131,072 ctx
Summary
User reports Qwen3.6 35B-A3B at 19.53 t/s generation and 132.58 t/s prompt processing on an RTX 4060 Ti 16GB with an i7 7th-gen CPU and 48GB DDR4.
Setup is llama.cpp (llama-server) with Q5_K_XL GGUF, q8_0 KV cache, 131072 context, flash attention on, and --cpu-moe keeping experts in system RAM; only 4.5GB of VRAM is occupied.
User compares this to a dense ~26B model at Q4_K_M with -ngl 25, which crawls at around 2 t/s, and asks why the MoE run is nearly twice as fast despite streaming expert layers over PCIe.