llamaperf

Gemma 2 9B

on Unknown GPU

Tone: positive
Oct 3, 2026
Throughput
8.2 t/s gen

Summary

User reports Gemma 9B at 8.2 t/s sustained on a Snapdragon 8 Gen 5 phone, running purely on the CPU with an ~8GB memory footprint. Setup is a custom engine with ARM-specific optimizations, chosen over mobile OpenCL/Vulkan drivers, plus an ONNX-based Kokoro TTS voice engine running natively on-device. The run is fully offline and air-gapped with zero API calls.

Compare this run

Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.

Every Gemma 2 report →