Gemma 4 31B

User reports Gemma 4 31B IT NVFP4 at 66.80 tok/s aggregate effective batch throughput on an RTX 5070 Ti 16GB. Setup is llama.cpp with NVFP4 GGUF weights, a Gemma 4 31B Assistant/MTP draft model, --spec-type draft-mtp, --spec-draft-n-max 5, flash attention on, and all layers offloaded to GPU. The benchmark used mtp-bench across 13 mixed tasks with 2,016 predicted tokens in 30.18 s and a 60.2% draft acceptance rate. Aggregate throughput rose from 53.10 tok/s at n_max 2 to 66.80 tok/s at n_max 5, while acceptance fell from 78.2% to 60.2%. Per-task speeds ranged from 35.8 tok/s on creative writing to 104.6 tok/s on repeated patterns. The author notes these are aggregate figures and that no non-speculative baseline was measured.