Gemma 4 5.1B E2B

on Unknown GPU

Sep 9, 2026

Throughput

25.9 t/s gen · 639.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.

Summary

User reports a custom C runtime for CPU inference with int8 weights, OpenMP, AVX2 and AVX-512 VNNI, running faster than llama.cpp on a Ryzen 7 7700.

View source ↗