DeepSeek V4 Flash 284B (13B active)
on NVIDIA RTX PRO 6000 Max-Q · llama.cpp · 200,000 ctx
Summary
User benchmarks multiple quants on a Bosgame M5 with an RTX PRO 6000 Max-Q eGPU.
Setup is a mixed GPU configuration with CUDA0 and ROCm0, using a DSpark drafter ported from a closed PR. UD-Q8_K_XL (150.8 GiB) reaches 44.0 t/s decode and 564 t/s prefill with draft acceptance 0.535. UD-Q4_K_XL (144.4 GiB) reaches 48.4 t/s decode and 585 t/s prefill with draft acceptance 0.532. UD-Q2_K_XL (90.2 GiB) reaches 59.5 t/s decode and 1513 t/s prefill with no drafter.