Qwen3.6 35B (3B active)
on NVIDIA Tesla P40 24GB · llama.cpp
Sep 28, 2026
Summary
User reports Qwen3.6 35B-A3B at IQ4 on a Tesla P40 24GB, getting 37-83 tok/s generation with MTP enabled.
Setup is llama.cpp with IQ4 quant. Prefill starts at 600 t/s for short prompts and degrades to 300-400 t/s by the end of a long prompt.
User asks whether IQ quants are actually slower on the P40, noting they did not feel a slowdown moving from Q4 to IQ4.