llamaperf

Qwen3.6 35B (3B active)

on NVIDIA RTX A2000 12GB · llama.cpp · 131,072 ctx

NVIDIA hardware
Sep 18, 2026
Throughput
26.1 t/s gen · 233.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
UD-Q5_K_M (GGUF)
KV cache
q8_0
MTP (Multi-Token Prediction)
on
System RAM
128 GB
VRAM reported
12 GB

Use cases

visionlong-context

Summary

User reports Qwen3.6-35B-A3B at 26.1 t/s generation and 233 t/s prompt eval at 131,072 context on an RTX A2000 12GB with a Xeon Silver 4314 and 128 GB DDR4. Setup is llama.cpp master with the unsloth UD-Q5_K_M GGUF, q8_0 KV cache, --n-cpu-moe 32, MTP, and mmap on. Empty-context generation was 35-38 t/s; the 118k-deep figure was measured at --n-cpu-moe 31. A 3000x2000 image request took 32 s, and the user notes that --n-cpu-moe 31 crashed on that image while 32 passed.