TensorSharp on the NVIDIA A40 48GB
llamaperf has 3 TensorSharp reports on the NVIDIA A40 48GB, none of them a plain single-card run with a model size and quant, so there is no median to quote. The reports are listed below with how each one ran.
3 TensorSharp reports on the NVIDIA A40 48GB
Newest first. A run on several cards, with speculative decoding, with part of the model in system RAM or with batched requests says so, since each describes a different setup.
- DeepSeek V4.1 Flash 552B · 16B active · Q2_K41.1 t/s(8 cards)
- DeepSeek V4.1 Flash 552B · 16B active · Q4_K_M32.5 t/s(8 cards, part in system RAM)
- DeepSeek V4 Flash 284B · 13B active · Q8_K_XL31.5 t/s(4 cards, part in system RAM)
Frequently asked
How fast is TensorSharp on the NVIDIA A40 48GB?
llamaperf has 3 TensorSharp reports on the NVIDIA A40 48GB, none of them a plain single-card run with a model size and quant, so there is no median to quote. The reports are listed below with how each one ran.