llamaperf

Nemotron 3.5 Lightning 30B (3B active)

on NVIDIA RTX 3090 · vLLM

Tone: positiveNVIDIA hardware
Sep 7, 2026
Quant
W4A16
VRAM reported
24 GB

Summary

User compares a W4A16 quant of Nemotron 3.5 Lightning 30B-A3B against an IQ4_XS GGUF on an RTX 3090. Setup is vLLM for W4A16 and llama.cpp for IQ4_XS. The two are near-parity in instruction following benchmarks, with about 4.5x throughput by B16. The user describes the model as fast and reliable, and suitable for batch labelling and agentic responses.