Ornith1.5 35B (3B active)
on NVIDIA Tesla P40 24GB · llama.cpp
Summary
User reports Ornith 1.5 35B-A3B at 60.8 t/s decode and 1104 t/s prefill at zero depth on a single Tesla P40 24GB.
Setup is a custom llama.cpp fork with Q4_0 weights, q8_0 KV cache, full 262k context, on Windows over PCIe x1; the llama-bench table is without MTP.
Decode falls to 46.0 t/s at 32k, 29.1 t/s at 100k and 18.8 t/s at 190k. With MTP (draft n=2) the real server gives 65-74 t/s on short prompts, and a 101k-token prompt reads at ~415 t/s and generates at 35-39 t/s. ECC off added 1.5 GB VRAM and ~9% decode; q4 KV gave 30.6 t/s versus 42.2 t/s for q8 at 32k.