Nemotron 3.5 Lightning 30B (3B active) Ternary
on Unknown GPU · llama.cpp · 368,000 ctx
Summary
User reports a ternary (1.58-bit) Nemotron 3.5 Lightning 30B-A3B GGUF running on a 12 GB GPU with 368K context, at 172 tok/s decode and 7000+ tok/s prefill.
Setup is stock llama.cpp (validated on b11555+, b11796) and also the PrismML fork / Bonsai (validated on b10793), one file for both engines.
The card does not name the specific GPU model.
Compare this run
Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.