DeepSeek V4.1 Flash 552B (16B active)
on 8× NVIDIA A40 48GB · TensorSharp
Sep 18, 2026
Summary
User reports DeepSeek V4.1 Flash at 41.1 tok/s decode on 8× NVIDIA A40 with layer-split execution.
Setup is the TensorSharp engine with Q2_K weights; prefill measured 532–541 tok/s with GPU-resident Engram, and cold model loading from MooseFS took 144–155 seconds.
Decode improved from 35.6 to 41.1 tok/s after reducing decode graph partitions from 565–577 to 8. These are project benchmarks, not a comparison against llama.cpp.