Nemotron 3.5 Super 120B (12B active) VL
on NVIDIA RTX Pro 6000 Blackwell · SGLang · 786,432 ctx
Use cases
long-context
Summary
User reports running Nemotron 3.5 Super VL 120B-A12B in NVFP4 on a single RTX PRO 6000 with SGLang, getting 150 to 250 t/s generation.
Setup uses a 3M token KV cache pool at fp8_e4m3 with a 786432 context length, allowing 4 concurrent requests of about 750k tokens each, with EAGLE speculative decoding (5 steps, 6 draft tokens).
The user notes the model uses no system memory lookup table and is slightly faster than before; CUDA graphs work for decode but not prefill.