Qwen3.6 35B (3B active)
on NVIDIA RTX 3080 Ti · llama.cpp · 131,072 ctx
Use cases
tool-uselong-context
Summary
User reports Qwen3.6-35B-A3B at 50.47 tok/s generation and 303.7 tok/s prompt evaluation on an RTX 3080 Ti 12 GB, with a 131072-token context.
Setup is llama.cpp b12530 with Q4_K_M GGUF, q8_0 keys and q4_0 values KV cache, Flash Attention on, one slot, and --n-cpu-moe 22 offloading experts to the 32 GB host RAM.
The server loaded in 12.6 s and used 11,817 MiB of GPU memory, leaving 270 MiB free; 4.9 GB system RAM remained available. Both OpenAI- and Anthropic-format tool calls passed.