llamaperf

Qwen3.6 35B (3B active)

on NVIDIA RTX 4060 Ti 16GB · llama.cpp · 131,072 ctx

NVIDIA hardware
Oct 17, 2026
Throughput
19.5 t/s gen · 132.6 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
Q5_K_XL (GGUF)
KV cache
q8_0
System RAM
48 GB
VRAM reported
16 GB

Summary

User reports Qwen3.6 35B-A3B at 19.53 t/s generation and 132.58 t/s prompt processing on an RTX 4060 Ti 16GB with an i7 7th-gen CPU and 48GB DDR4. Setup is llama.cpp (llama-server) with Q5_K_XL GGUF, q8_0 KV cache, 131072 context, flash attention on, and --cpu-moe keeping experts in system RAM; only 4.5GB of VRAM is occupied. User compares this to a dense ~26B model at Q4_K_M with -ngl 25, which crawls at around 2 t/s, and asks why the MoE run is nearly twice as fast despite streaming expert layers over PCIe.