Sep 7, 2026
Summary
User compares a W4A16 quant of Nemotron 3.5 Lightning 30B-A3B against an IQ4_XS GGUF on an RTX 3090.
Setup is vLLM for W4A16 and llama.cpp for IQ4_XS.
The two are near-parity in instruction following benchmarks, with about 4.5x throughput by B16. The user describes the model as fast and reliable, and suitable for batch labelling and agentic responses.