llamaperf

Inkling 2 Mini 26B (3.4B active)

on NVIDIA RTX 5090 · NInfer · 10,000 ctx

Tone: positiveNVIDIA hardware
Nov 21, 2026
Throughput
182.0 t/s gen · 6664.0 tokens/s prompt processing (prefill)Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.
Quant
UD-IQ4_XS (GGUF)
VRAM reported
32 GB

Use cases

vision

Summary

User reports Inkling 2 Mini at 182 t/s decode and 6,664 t/s prefill on an RTX 5090, compared with 97 t/s decode and 2,318 t/s prefill for llama.cpp on the same GGUF. Setup is the user's NInfer-all fork with Unsloth's UD-IQ4_XS GGUF, a 10K-token prompt, and the DSpark drafter enabled; llama.cpp was Unsloth's build because mainline does not load this GGUF. The model does not fit in 16 GB or less, so on those cards experts sit in system RAM and the GPU caches the most-used ones; reading experts straight from disk is slow, about 9 t/s on the RTX 5060. The post also lists results for the RTX 4090, 3090, 5080, 3060 and 5060.

Compare this run

Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.

Inkling 2 Mini 26B · 3.4B active at 4-bit by hardware

Bars are medians on one scale, 0 to 108.9 tokens/s. Press ▶ to watch text arrive at a row's speed.

Models most run on the NVIDIA RTX 5090

Bars are medians on one scale, 0 to 563.1 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.

Everything reported on the NVIDIA RTX 5090 →Every Inkling 2 Mini report →Estimate Inkling 2 Mini on this card →