mixed Q4K-F16-Q8

on 2× AMD Strix Halo 128GB · DS4 ds4-bench-tp / ROCm 7.14 / gfx1151 / 71e6a24

Oct 13, 2026

Throughput

19.9 t/s gen · 270.1 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.

Quant

Q4_K experts / mixed F16-Q8 (GGUF)

MTP (Multi-Token Prediction)

off

System RAM

256 GB

Summary

User reports DeepSeek V4 Flash at 19.9 t/s generation and 270.1 t/s prompt processing on two AMD Strix Halo nodes with 128 GB RAM each, linked by OdinLink USB4/TB5 RDMA. Setup is the Antirez Q4_K experts build with F16 HC/compressor/indexer and Q8 attention/shared/output, tensor parallel across two Radeon 8060S iGPUs with a balanced 50/50 expert split. The run used a fixed 2,048-token prompt followed by 300 generated tokens, with ordinary decode and no speculative decoding; the source gives 19.89 t/s and 270.10 t/s exactly.