llamaperf
Submit

Nemotron-Nano v2 9B

on NVIDIA RTX PRO 6000 Max-Q · llama.cpp

Family: Nemotron-Nano v2
NVIDIA hardware
Sep 7, 2026
Share
Throughput
16384.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.

Summary

User benchmarks llama.cpp PR #22675 for Mamba-2 prefill acceleration on Nemotron-Nano-9B-v2, with pp16384 tok/s reported. The PR is tested at various ub values. The user also mentions DeepSeek-V4-Flash-UD-IQ2_XXS on an M4 Max for a Metal FWHT kernel PR, but the primary benchmarked model is Nemotron-Nano-9B-v2.
View source ↗