llamaperf

Inkling 2 Mini 26B (3B active) abliterated

on NVIDIA RTX 4090 · Xolotl · 8,192 ctx

Tone: positiveNVIDIA hardware
Nov 21, 2026
Throughput
59.7 t/s gen · 1162.0 tokens/s prompt processing (prefill)Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.
Quant
XQ (GGUF)
VRAM reported
24 GB

Summary

User reports Inkling 2 Mini at 59.7 t/s decode and 1,162 t/s prefill at 8K on a single RTX 4090. Setup is a custom engine written in Xolotl with a 13.9 GB GGUF quant of the abliterated model, which fits on the 24 GB card. The model is a 26B MoE with about 3B active parameters and a looped middle of 12 layers run 3 times per token. Held-out KL against the parent is 0.013 with 96.3% top-1 agreement; refusals were 4/100 on a harmful prompt set and 1/100 on harmless ones. The model's MTP drafter is an optional separate file and was not used for these figures. User expects 100+ t/s after fusing the many small per-token kernels.