DeepSeek V4.1 Flash 552B (16B active)
on DGX Station GB300 · vLLM
Summary
User reports DeepSeek-V4.1-Flash M3 at 258-266 tok/s per user at C1 on a DGX Station GB300 with an RTX PRO 6000 Blackwell Max-Q expert sidecar.
Setup is vLLM with NVFP4 Engram tables and NVFP4 KV cache, 265 hot experts on the GB300 and 3,960 cold experts on the sidecar, sharing the station with a video model and keeping 3.42M KV tokens.
Aggregate throughput is 1,046 tok/s at C8 and 1,629 tok/s at C16, up from 583 and 821 without the sidecar; prefill is 44.5K tok/s (60K on real text). GSM8K-200 is 98.0% and needle passes at 114K.