DeepSeek V4 Pro

User benchmarks DeepSeek V4 Pro GGUF at 794 GB on an Epyc 9374F with 12x96 GB DDR5 and an RTX PRO 6000 Max-Q. Setup is llama.cpp on a branch with expert offloading. Prompt processing ranges from 192 t/s at 8K context to 66 t/s at 1M context, and generation ranges from 11.73 t/s to 5.83 t/s. RAM usage is 69.3% of 1152 GB and VRAM usage is 78986 MiB of 96 GB, with power around 500 W during prompt processing. The user notes mainline llama.cpp issues with memory waste, a broken quantized KV cache, and bugs with prompt cache reuse.

generation:

7.1 tokens/s

prompt processing (prefill):

46.2 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User benchmarks at various context depths on an Epyc 9374F. GPU VRAM usage is 90815MiB/97887MiB and RAM usage is 907.5GB/1152GB. Power draw is about 100W prefill and 150W token generation for the GPU, and about 400W for the CPU and motherboard. The original model files are used with no conversion.