Qwen3-Next 80B (3B active)
on Intel Arc iGPU · Janas-LLM · 1,024 ctx
Summary
User reports Qwen3-Next-80B-A3B at 26.4 tok/s generation and 68.4 tok/s prompt processing on a laptop with an Intel Core Ultra 9 185H, 32 GB DDR5, integrated Arc GPU and NPU, and NVMe Gen4.
Setup is the author's Janas-LLM engine in plain C, streaming MoE experts from NVMe with an expert cache in RAM; replies run on the CPU, prompts on the iGPU or NPU. The 48.4 GB model is larger than system RAM.
llama.cpp on the same files reached 3.0 tok/s generation and 17.7 tok/s prompt on this model. The post also tables nine other models, MTP drafts giving up to 63% faster chat replies on Qwen3.8-Flash-Next, and 18.2 tok/s generation at 8,192 tokens on Qwen3-4B with an 8-bit KV cache.
Compare this run
Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.
Qwen3-Next 80B · 3B active at 4-bit by hardware
- 116.0 t/s · one run
Bars are medians on one scale, 0 to 116.0 tokens/s. Press ▶ to watch text arrive at a row's speed.