llamaperf

Qwen3.6 35B (3B active)

on NVIDIA Tesla P40 24GB · llama.cpp

Tone: mixedNVIDIA hardware
Sep 28, 2026
Throughput
37-83 t/s gen
Quant
IQ4 (GGUF)
VRAM reported
24 GB

Summary

User reports Qwen3.6 35B-A3B at IQ4 on a Tesla P40 24GB, getting 37-83 tok/s generation with MTP enabled. Setup is llama.cpp with IQ4 quant. Prefill starts at 600 t/s for short prompts and degrades to 300-400 t/s by the end of a long prompt. User asks whether IQ quants are actually slower on the P40, noting they did not feel a slowdown moving from Q4 to IQ4.