llamaperf
Oct 17, 2026
Throughput
50.0 t/s gen
Quant
Q4

Summary

User asks for a Vulkan/radv-based inference engine that reaches over 1400 tok/s prefill for Qwen3.8 Flash-Next Q4, and reports their own tiny engine gets around 50 tok/s decode but prefill stuck below 1k tok/s. They suspect the gap is the Vulkan versus HIP/ROCm implementation. No GPU or system memory is named.