llamaperf

Inkling-2 Mini

on NVIDIA RTX 5060 Ti 16GB · llama.cpp · 150,000 ctx

Tone: positiveNVIDIA hardware
Nov 21, 2026
Throughput
18-23 t/s gen · 283.0 tokens/s prompt processing (prefill)Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.
Quant
UD-Q4_K_M (GGUF)

Use cases

codingagentic

Summary

User reports Inkling-2-Mini completing a Godot collision-shape task in 23 minutes on an RTX 5060 Ti, with prefill at 283 tokens/s and generation at 18-23 tokens/s. Setup is llama.cpp with UD-Q4_K_M GGUF, 150000 context, 14 MoE layers on CPU (--n-cpu-moe 14), and all GPU layers offloaded. The user notes Qwen3.8 Next Flash on an R9700 ran at 30-40 tokens/s and GLM 5.3 Flash Maya-S at 7-10 tokens/s, but neither finished the task in 1-2 hours. They also recall 50-60 tokens/s on Qwen3.6 35B with a Q8 KV cache, and wish Inkling-2-Mini generated faster.