llamaperf
Sep 7, 2026
Throughput
29.0 t/s gen · 1328.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
Q4
VRAM reported
96 GB

Summary

User reports DeepSeek V4 Flash 0731 (Q4) at ~29 tok/s decode on one RTX PRO 6000 96GB. Setup is Krasis v1.0.19 with INT4 experts, keeping the hottest 6,440 of 11,008 routed experts in VRAM and serving the rest from system RAM during decode. Decode after a roughly 1K prompt was 29.4, 28.2 and 28.5 tok/s for 50, 100 and 250 tokens, and 19.4 tok/s after a 62K prompt. Prefill reaches 1,328 tok/s at 23,348 tokens and 1,204 tok/s at 62,403, in internal timing-disabled results rather than HTTP round trips.