llamaperf

Unknown family

on 2× NVIDIA Tesla P100 16GB

Tone: mixedNVIDIA hardware
Nov 21, 2026
Throughput
38.0 t/s gen

Summary

User reports about 38 t/s average generation on 2x P100, with 60+ t/s on some tasks when MTP is enabled. Prefill is about 200 t/s, which the user calls painfully slow; the system draws 500W under load. The user says it works but is basically unusable for anything interactive and asks whether prefill can be sped up on this old hardware.

Compare this run

Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.

Models most run on the NVIDIA Tesla P100 16GB

Bars are medians on one scale, 0 to 10.0 tokens/s. Press ▶ to watch text arrive at a row's speed.

Everything reported on the NVIDIA Tesla P100 16GB →