llamaperf
Nov 14, 2026
Throughput
39.0 t/s gen · 1161.0 tokens/s prompt processing (prefill)Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.
Quant
Q4
System RAM
128 GB

Summary

User reports Qwen3.8 Flash-Next at 39.0 tok/s per request on two Strix Halo 128GB boxes with tensor parallelism, against 29.8 tok/s on one box. Setup is a fork of gufo with Q4 weights (111 GB), TP=2 over USB4 RDMA, each box holding 57 GB of the model. Decode is plain autoregressive with no speculation. At 2 concurrent requests the two-box run gives 31.7 tok/s versus 23.3 on one box, and at 4 concurrent 24.6 versus 18.0. Short-prompt prefill is 1161 tok/s on two boxes and 1155 on one.

Compare this run

Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.

Qwen3.8 125B · 6B active at 4-bit by hardware

Bars are medians on one scale, 0 to 232.0 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.

Models most run on the AMD Strix Halo 128GB

Bars are medians on one scale, 0 to 77.0 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.

Everything reported on the AMD Strix Halo 128GB →Every Qwen3.8 report →Estimate Qwen3.8 on this card →