Qwen3.5 0.8B
on Unknown GPU · qwen35-cpu
Summary
User reports a custom engine and quant format for CPU inference at 122.9 t/s decode at batch size 1 and 642 t/s at batch size 16.
The run is compared against llama.cpp and ik_llama.cpp.
It measures PPL 15.80 and KL 0.0602.