Qwen3.6 35B (3B active) Ornith-Agents-A1-3.6
on NVIDIA RTX 6000 Ada · llama.cpp · 131,072 ctx
Summary
User reports Ornith-Agents-A1-3.6-35B-A3B (a dare_ties merge on Qwen3.6-35B-A3B with a grafted MTP block) at up to ~260 tok/s on an RTX 6000 Ada with 96GB VRAM.
Setup is llama.cpp with Q4_K_M GGUF, q8_0 KV cache, 131072 context, --parallel 4, and native MTP speculative decoding (--spec-type draft-mtp, --spec-draft-n-max 4). Model used ~25 GB of the 95.6 GB VRAM.
MTP draft acceptance stayed in the 42-48% range across generation lengths; measured throughput converged to ~245-260 tok/s. Q8_0 with MTP gave 190 tok/s, MTP disabled gave 150-200 tok/s, and vLLM measured 130 tok/s. spec-draft-n-max 3 gave 270 tok/s and spec-draft-n-max 2 gave 277 tok/s.