llamaperf

Nemotron 3.5 Bolt 74B (9B active)

on NVIDIA RTX 5090 · llama.cpp · 32,768 ctx

Tone: mixedNVIDIA hardware
Oct 20, 2026
Throughput
343.0 t/s gen
Quant
IQ2_XXS (GGUF)
KV cache
q8_0
VRAM reported
32 GB

Use cases

agentictool-usecoding

Summary

User reports Nemotron 3.5 Bolt 74B-A9B at 343 t/s generation on an RTX 5090 32GB, median of 15 runs (303 to 354). Setup is llama.cpp b11750 with NVIDIA's IQ2_XXS GGUF, q8_0 KV cache, 32K context, single stream, and the LiLiCorr drafter at 55% acceptance; 25.0 GiB of VRAM with the drafter and projector. Without the drafter it runs 166 t/s at 22.7 GiB, and 21.0 GiB without drafter or projector. llama-bench gives pp512 5,166 t/s and tg128 172 t/s. On a 33-task DevOps tool-use eval it passed 26 of 33 (79%, score 0.88) with zero safety failures, 54% unrecorded tool calls and 28.3 GiB VRAM at four slots.