DeepSeek V4 Pro 1600B (49B active)
on NVIDIA RTX PRO 6000 Max-Q · llama.cpp · 1,048,576 ctx
Summary
User benchmarks DeepSeek V4 Pro GGUF at 794 GB on an Epyc 9374F with 12x96 GB DDR5 and an RTX PRO 6000 Max-Q.
Setup is llama.cpp on a branch with expert offloading. Prompt processing ranges from 192 t/s at 8K context to 66 t/s at 1M context, and generation ranges from 11.73 t/s to 5.83 t/s. RAM usage is 69.3% of 1152 GB and VRAM usage is 78986 MiB of 96 GB, with power around 500 W during prompt processing.
The user notes mainline llama.cpp issues with memory waste, a broken quantized KV cache, and bugs with prompt cache reuse.