Nemotron 3.5 74B (9B active) Bolt

User benchmarks Nemotron 3.5 Bolt (74B, 2-bit) and Qwen3.8 27B (Q4_K_M) on an RTX 3090, comparing context limits and speed at various fill depths. Setup is llama.cpp with Q8_0 KV cache, flash attention on, and no CPU offload. Bolt fits its full 256K context at 21.8GB VRAM, while Qwen3.8 27B maxes out at about 216K (221,184 tokens) at 23.2GB. At empty context, Bolt decodes at 86.6 t/s and Qwen at 42.3 t/s. At 128K fill, Bolt decodes at 52.2 t/s and Qwen at 23.3 t/s. Prompt processing is faster for Qwen at empty context (1237 vs 822 t/s), but Bolt is faster from 64K onward. The user notes that ubatch 128 was carried over from a previous test, so prompt numbers are lower than a tuned setup would achieve.