llamaperf

MiniCPM-V 4.7 35B (3B active)

on NVIDIA V100 16GB · llama.cpp · 131,072 ctx

Tone: positiveNVIDIA hardware
Oct 17, 2026
Throughput
93.0 t/s gen
Quant
mix_q3gud_im (GGUF)
KV cache
q4_0
VRAM reported
16 GB

Use cases

visionlong-contextcodingagentic

Summary

User reports MiniCPM-V 4.7 35B-A3B at about 93 tok/s on a Tesla V100 16GB under Windows 11, with vision and 128k context. Setup is stock llama.cpp with OpenBMB's official mix_q3gud_im quant (15.5 GB), q4_0 KV cache, flash attention on, and the vision encoder kept on CPU so all MoE experts fit on the GPU. A 120k-token prompt was answered correctly without a crash, using 16.17 of 16.38 GB VRAM. At 262k context generation drops to 70 tok/s. DSpark n-gram speculative decoding was slower because the drafter's VRAM pushed experts back to RAM. The user also benchmarked 15 models on a 72-task code-graded suite, where this model scored 87.5 overall.