DeepSeek V4 Flash 284B (13B active)
on 4× AMD Radeon AI PRO R9700 32GB · affinity
Summary
User reports DeepSeek-V4-Flash-0731 at 79 tok/s greedy decode on four Radeon AI PRO R9700 cards.
Setup is the affinity engine with a prebuilt 2.875 bpw quant, all 11,008 experts resident in VRAM with no host pool allocated, on a Threadripper PRO 9955WX with 128 GB RAM.
The same box on two cards gave 36 tok/s with 45% of experts in VRAM and the rest streaming from a 51 GB pinned host pool. Sampled decode at temp 1.0 was 63 tok/s and about 90 tok/s on a chat answer with thinking on. Load takes about a minute, 42 seconds of it pushing 96 GB of experts onto the cards.