llamaperf

DeepSeek V4.1 Flash 552B (16B active)

on 5× NVIDIA A40 48GB · TensorSharp · 1,800 ctx

NVIDIA hardware
Oct 24, 2026
Throughput
35.5 t/s gen · 960.5 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
UD-IQ2_XXS (GGUF)

Use cases

summarization

Summary

User reports DeepSeek V4.1 Flash at 35.5 t/s decode after a ~1.8K-token prompt on 5x NVIDIA A40, with prefill at 960.5 t/s. Setup is TensorSharp with UD-IQ2_XXS GGUF weights, tensor parallel across five GPUs, thinking disabled and greedy decoding; the 125.9 GiB Engram table stayed in host memory, so the run used both GPU and system memory. Decode was 35-36 t/s without a drafter and 51-55 t/s with the DSpark Q8_0 drafter; the same machine's llama.cpp fork reached 29.6 t/s decode and 174.6 t/s prefill. Disabling the approximate YOCO prefill optimization gave 563.9 t/s prefill and perplexity 2.4811 versus 2.4978 with it on and 2.4697 for llama.cpp. A four-request batch took 21.2-25.5 s on TensorSharp and 43.3-46.8 s on llama.cpp.