llamaperf

DeepSeek V4.1 Flash 552B (16B active)

on 8× NVIDIA A40 48GB · TensorSharp

NVIDIA hardware
Sep 18, 2026
Throughput
41.1 t/s gen · 532-541 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
Q2_K (GGUF)

Summary

User reports DeepSeek V4.1 Flash at 41.1 tok/s decode on 8× NVIDIA A40 with layer-split execution. Setup is the TensorSharp engine with Q2_K weights; prefill measured 532–541 tok/s with GPU-resident Engram, and cold model loading from MooseFS took 144–155 seconds. Decode improved from 35.6 to 41.1 tok/s after reducing decode graph partitions from 565–577 to 8. These are project benchmarks, not a comparison against llama.cpp.