MiniCPM-V 4.7 35B (3B active)
Unknown GPU
- generation:
- 13.0 tokens/s
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports MiniCPM-V 4.7, a 35B-A3B MoE vision model, running at 13+ tok/s on an iPhone 17 Pro Max with a 3GB inference memory footprint. Experts are streamed from flash storage using Lookahead Expert Scheduling (LES), which predicts upcoming expert needs and overlaps loading with computation. The 3GB figure is the inference memory footprint; the app README lists a ~16.6GB download and recommends at least 12GB of device RAM. A smaller 1B version exists, and code and weights are Apache 2.0.