llamaperf
Oct 9, 2026
Throughput
34.2 t/s gen · 1541.2 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
UD-Q4_K_XL (GGUF)
MTP (Multi-Token Prediction)
off

Summary

User reports Qwen3.8-Flash-Next at 34.19 t/s decode and 1541.16 t/s prefill on two AMD Strix Halo 128GB hosts, one Radeon 8060S gfx1151 GPU per host, tensor parallel world size 2 over RDMA/InfiniBand. Setup is the Gufo RDMA fork with Unsloth UD-Q4_K_XL GGUF (four shards), configured context capacity 262144 tokens, greedy temperature 0, thinking off, speculative decoding and MTP off; the measured input was 2040 tokens, not 262144. Single warmed prose sample, one session, HTTP OpenAI chat-completions SSE; source-maintainer measurement, not independently replicated, with no standard deviation. KV-cache precision, Flash Attention setting, measured resident RAM and verified binary identity are unknown; the prose labels revision ea76571 while the raw artifact records 795407e30460aeb730aaff2316739baa3a1348e2 with dirty=true.