Inkling 2 Mini
NVIDIA RTX 5060 Ti 16GB · llama.cpp · 128,000 ctx
- generation:
- 44.4 tokens/s
- quant:
- IQ4_NL (GGUF)
- kv:
- q4_0
User reports Inkling 2 Mini at 44.35 t/s on an RTX 5060 Ti 16GB with 32 GB system RAM. Setup is llama.cpp (build 19bc41dbb) with the IQ4_NL GGUF from ggml-org, q4_0 KV cache, 128k context, no vision, and ngram-mod speculation. The same setup with q8_0 KV cache reached 43.34 t/s; the user started at 12 t/s before tuning.