Qwen3.6 35B (3B active)
on AMD Strix Halo 128GB · vLLM · 2,048 ctx
Use cases
long-context
Summary
User reports Qwen3.6-35B-A3B at 3,581 t/s prefill and 32.6 t/s decode at 2k context on a Strix Halo 128GB APU, comparing stock vLLM against the same vLLM with the libstrix HIP kernel plugin.
Setup is vLLM with bf16 weights and bf16 KV cache, single stream on one APU; the plugin installs via pip and only activates on ROCm gfx1151.
At 64k the same run goes from 260 to 2,617 t/s prefill and 12.8 to 26.9 t/s decode; at 128k with int8 KV it reaches 1,751 t/s prefill and 26 t/s decode, and a 256k prompt prefills in 241 s versus 7,568 s stock. KLD against stock is 0.001-0.006 with 97-99% top-1 agreement, and the user notes decode gains are absent where the replaced kernels are not the bottleneck.