Nemotron 3.5 Super 120B (12B active) VL
NVIDIA RTX Pro 6000 Blackwell · SGLang · 786,432 ctx
- generation:
- 150-250 tokens/s
- quant:
- NVFP4 (NVFP4)
- kv:
- fp8_e4m3
User reports running Nemotron 3.5 Super VL 120B-A12B in NVFP4 on a single RTX PRO 6000 with SGLang, getting 150 to 250 t/s generation. Setup uses a 3M token KV cache pool at fp8_e4m3 with a 786432 context length, allowing 4 concurrent requests of about 750k tokens each, with EAGLE speculative decoding (5 steps, 6 draft tokens). The user notes the model uses no system memory lookup table and is slightly faster than before; CUDA graphs work for decode but not prefill.