Unknown family
on 2× NVIDIA Tesla P100 16GB
Summary
User reports about 38 t/s average generation on 2x P100, with 60+ t/s on some tasks when MTP is enabled.
Prefill is about 200 t/s, which the user calls painfully slow; the system draws 500W under load.
The user says it works but is basically unusable for anything interactive and asks whether prefill can be sped up on this old hardware.
Compare this run
Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.
Models most run on the NVIDIA Tesla P100 16GB
- Qwen3.8 27B3-bit10.0 t/s · one run
Bars are medians on one scale, 0 to 10.0 tokens/s. Press ▶ to watch text arrive at a row's speed.