Nemotron 3.5 Lightning 30B (3B active) Ternary
Unknown GPU · llama.cpp · 368,000 ctx
- generation:
- 172.0 tokens/s
- prompt processing (prefill):
- 7000.0 tokens/s
- quant:
- ternary 1.58-bit (GGUF)
User reports a ternary (1.58-bit) Nemotron 3.5 Lightning 30B-A3B GGUF running on a 12 GB GPU with 368K context, at 172 tok/s decode and 7000+ tok/s prefill. Setup is stock llama.cpp (validated on b11555+, b11796) and also the PrismML fork / Bonsai (validated on b10793), one file for both engines. The card does not name the specific GPU model.