Qwen3.6 35B (3B active)
on NVIDIA RTX 5070 Ti · llama.cpp · 65,536 ctx
Oct 7, 2026
Summary
User reports Qwen3.6-35B-A3B at 111.72 tok/s decode and 4,326.76 tok/s prompt eval on an RTX 5070 Ti 16GB.
Setup is llama.cpp with Q4_K_M GGUF, F16 KV cache, 65,536 context, cache 132 and ub=8192, single request.
Peak VRAM was 14,369 MiB. At NP4 the same model reached 247.79 tok/s aggregate decode (62.00-67.78 per request) and 1,142.11-2,243.00 tok/s prefill per request. Gemma-4-26B-A4B-it Q4_0 runs are also listed, reaching 104.58 tok/s at NP1 and 175.56 tok/s aggregate at NP16.