Qwen3.8 Flash-Next / EngramHalo / 1 active request, 8 slots / MTP and ngram on
on AMD Strix Halo 128GB · EngramHalo.cpp · 16,384 ctx
Summary
User reports Qwen3.8 at 35.9 t/s generation and 125.7 t/s prompt processing on an AMD Strix Halo 128GB machine.
Setup is EngramHalo with UD-Q4_K_XL, q8_0 target K/V cache, Flash Attention, 16384 tokens per slot and eight slots, with draft-mtp plus ngram-mod speculative decoding.
The 35.9 t/s is the server's native TG value for the first one-user request, 71 input and 256 output tokens; client aggregate throughput at one user is 33.2 t/s.