llamaperf

Qwen3.6 35B (3B active)

on NVIDIA Tesla P40 24GB · llama.cpp

NVIDIA hardware
Sep 30, 2026
Throughput
37.9 t/s gen · 982.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
Q3_K_XL (GGUF)
VRAM reported
24 GB

Summary

User reports Qwen3.6-35B-A3B at Q3_K_XL generating 37.9 tok/s on a single Tesla P40 24GB under WSL2 with driver 582.70 in the MCDM driver model. Setup is llama.cpp built with CUDA arch 61, GGML_CUDA_FORCE_MMQ=ON and GGML_CUDA_F16=OFF; llama-bench pp512 measures 982 tok/s prefill. With MTP speculative decoding at --spec-draft-n-max 2 generation rises to 51.6 tok/s (n-max 3: 48.3, n-max 4: 41.9). Prefill falls to about 790 tok/s at 6K tokens, 422-609 at 25K and about 216 at 78K. With MTP n-max 2, 4096 context with f16 KV uses 21710 MiB at 51.45 tok/s, 32768 with q8_0 KV uses 22086 MiB at 50.79 tok/s, and 131072 with q8_0 KV uses 22782 MiB at 50.54 tok/s.