Qwen3.6 35B (3B active)
on NVIDIA Tesla P40 24GB · llama.cpp
Sep 30, 2026
Summary
User reports Qwen3.6-35B-A3B at Q3_K_XL generating 37.9 tok/s on a single Tesla P40 24GB under WSL2 with driver 582.70 in the MCDM driver model.
Setup is llama.cpp built with CUDA arch 61, GGML_CUDA_FORCE_MMQ=ON and GGML_CUDA_F16=OFF; llama-bench pp512 measures 982 tok/s prefill.
With MTP speculative decoding at --spec-draft-n-max 2 generation rises to 51.6 tok/s (n-max 3: 48.3, n-max 4: 41.9). Prefill falls to about 790 tok/s at 6K tokens, 422-609 at 25K and about 216 at 78K. With MTP n-max 2, 4096 context with f16 KV uses 21710 MiB at 51.45 tok/s, 32768 with q8_0 KV uses 22086 MiB at 50.79 tok/s, and 131072 with q8_0 KV uses 22782 MiB at 50.54 tok/s.