Qwen3.6 35B (3B active)
on NVIDIA Tesla P40 24GB · colibri
Summary
User reports Qwen3.6-35B-A3B at 9.67 tok/s decode on a single Tesla P40 24GB, with a cold run at 9.63 tok/s.
Setup is colibri's CUDA backend built for sm_61, int4 gs=64 weights, 200 tokens greedy, single stream, whole model resident in VRAM (10240/10240 experts, 100% hit, 0 CPU misses).
The P40 is slower than the P100 once warm (9.67 vs 12.02 tok/s) on ~346 GB/s GDDR5 versus ~732 GB/s HBM2; the cold-to-warm gap is 0.4% versus the P100's 61%. Probe-off control gives 9.62 tok/s and byte-identical output.