llamaperf

Nemotron 3.5 Super 120B (12B active) VL

on NVIDIA RTX Pro 6000 Blackwell · SGLang · 786,432 ctx

Tone: positiveNVIDIA hardware
Oct 20, 2026
Throughput
150-250 t/s gen
Quant
NVFP4 (NVFP4)
KV cache
fp8_e4m3

Use cases

long-context

Summary

User reports running Nemotron 3.5 Super VL 120B-A12B in NVFP4 on a single RTX PRO 6000 with SGLang, getting 150 to 250 t/s generation. Setup uses a 3M token KV cache pool at fp8_e4m3 with a 786432 context length, allowing 4 concurrent requests of about 750k tokens each, with EAGLE speculative decoding (5 steps, 6 draft tokens). The user notes the model uses no system memory lookup table and is slightly faster than before; CUDA graphs work for decode but not prefill.