Gemma 2 9B
on Unknown GPU
Oct 3, 2026
Summary
User reports Gemma 9B at 8.2 t/s sustained on a Snapdragon 8 Gen 5 phone, running purely on the CPU with an ~8GB memory footprint.
Setup is a custom engine with ARM-specific optimizations, chosen over mobile OpenCL/Vulkan drivers, plus an ONNX-based Kokoro TTS voice engine running natively on-device.
The run is fully offline and air-gapped with zero API calls.