Gemma 2 9B
on Unknown GPU
Summary
User reports Gemma 9B at 8.2 t/s sustained on a Snapdragon 8 Gen 5 phone, running purely on the CPU with an ~8GB memory footprint.
Setup is a custom engine with ARM-specific optimizations, chosen over mobile OpenCL/Vulkan drivers, plus an ONNX-based Kokoro TTS voice engine running natively on-device.
The run is fully offline and air-gapped with zero API calls.
Compare this run
Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.