Qwen3.8 27B
on 2× NVIDIA Tesla P100 16GB · llama.cpp
Summary
User reports Qwen3.8-27B Q4_0 at 33.28 t/s decode and 406.9 t/s prefill on 2x Tesla P100 16GB with patched llama.cpp, up from 23.43 t/s decode and 233.6 t/s prefill on stock.
Setup is llama.cpp v0.7.0 with 41 sm_60 patches, tensor split across two P100s, 5 interleaved rounds per config.
The patch set also gives 77.6 t/s decode on Ornith-1.0-35B-A3B UD-Q4_K_M with tensor split, and layer split gives 20.68 t/s decode on the 27B. Tested only on 2x P100; some patches are generic for all cards.