Qwen3.6 35B (3B active)
on NVIDIA RTX 5090 · NInfer
Summary
User reports Qwen3.6-35B-A3B at 389.0 t/s decode and 5,310 t/s prefill on a single RTX 5090.
Setup is the NInfer engine (ninfer-ext fork) with no quant code stated, one model at a time, no Python in the inference path.
Decode figures are aggregate rates for the stated number of lanes in flight, not per-request latency; the C=1 prefill figure is a true per-request rate while C=2 and above are point aggregates that include each request's decode time. MTP rows draft 3 tokens with the full proposal head and reach 650.3 t/s at C=1, 958.0 at C=2, 1,163.9 at C=4 and 1,368.0 at C=8. The same chart also covers Gemma 4 26B-A4B, Gemma 4 31B IT M1, Qwen3.6-35B-A3B nvfp4, Qwen3.8-27B groupwise-int, Qwen3.8-27B nvfp4 and Qwen3.8-Flash-Next nvfp4.