llamaperf
Oct 5, 2026
Throughput
9.7 t/s gen
Quant
int4 gs=64
System RAM
128 GB
VRAM reported
24 GB

Summary

User reports Qwen3.6-35B-A3B at 9.67 tok/s decode on a single Tesla P40 24GB, with a cold run at 9.63 tok/s. Setup is colibri's CUDA backend built for sm_61, int4 gs=64 weights, 200 tokens greedy, single stream, whole model resident in VRAM (10240/10240 experts, 100% hit, 0 CPU misses). The P40 is slower than the P100 once warm (9.67 vs 12.02 tok/s) on ~346 GB/s GDDR5 versus ~732 GB/s HBM2; the cold-to-warm gap is 0.4% versus the P100's 61%. Probe-off control gives 9.62 tok/s and byte-identical output.