llamaperf

Inkling-2

5 reports

Run Inkling-2 yourself? Add your numbers.

Your coding agent can run a speed test on your machine and send it here with a free account. Or paste a result you've already posted.

Reported performance for Inkling-2

Filter this model’s reports by setup →
Tone: mixedAMD hardware
generation:
30.0 tokens/s
quant:
Q8_0 (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Inkling-2-Mini at about 30 tok/s on a single Radeon AI PRO R9700, which they call pretty slow. Setup is llama.cpp built from source with the Q8_0 GGUF from ggml-org; the model only runs on one card. The unsloth Q8_K_XL build did not work with the latest llama.cpp, and its Mainline_Support variant produced gibberish.

Nov 21, 2026

Inkling-2 Mini

Strix Halo 8060S 128GB · Loom

Tone: positive
generation:
44.8 tokens/s
prompt processing (prefill):
1427.0 tokens/s
quant:
UD-Q4_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Inkling-2-Mini at 44.8 t/s decode on a Strix Halo 8060S with 128 GB, against 23.8 t/s for llama.cpp ROCm and 20.8 t/s for llama.cpp Vulkan on the same GGUF weights. Setup is Loom with the unsloth UD-Q4_K_XL GGUF, single stream; prefill at ~1k is 1,427 t/s versus 356 t/s for llama.cpp ROCm. Decode is said to reach ~80% of the memory-bandwidth roofline (228 GB/s ÷ ~4 GB read per token ≈ 56 tok/s). At ~2k context decode is 40.8 t/s and prefill 1,496 t/s; prefill at ~9k is 1,550 t/s and at ~18k 1,465 t/s. Four streams aggregate 100.8 t/s. The package was produced by Gemini 4 Neon driving the loom-skills playbook in about 4.3 hours and ~490 tool calls, gated against llama.cpp with token-level goldens up to 37k tokens. Speculative decoding with dspark is still a work in progress.

Nov 21, 2026

Inkling-2 Mini

NVIDIA RTX 5060 Ti 16GB · llama.cpp · 150,000 ctx

Tone: positiveNVIDIA hardware
generation:
18-23 tokens/s
prompt processing (prefill):
283.0 tokens/s
quant:
UD-Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

codingagentic

User reports Inkling-2-Mini completing a Godot collision-shape task in 23 minutes on an RTX 5060 Ti, with prefill at 283 tokens/s and generation at 18-23 tokens/s. Setup is llama.cpp with UD-Q4_K_M GGUF, 150000 context, 14 MoE layers on CPU (--n-cpu-moe 14), and all GPU layers offloaded. The user notes Qwen3.8 Next Flash on an R9700 ran at 30-40 tokens/s and GLM 5.3 Flash Maya-S at 7-10 tokens/s, but neither finished the task in 1-2 hours. They also recall 50-60 tokens/s on Qwen3.6 35B with a Q8 KV cache, and wish Inkling-2-Mini generated faster.

Nov 21, 2026

Inkling-2 Mini

NVIDIA RTX 4090 · vLLM · 35,000 ctx

Tone: positiveNVIDIA hardware
generation:
190-220 tokens/s
quant:
NVFP4 experts (W4A16) + FP8 W8A16 attention/head/embeddings (NVFP4)
kv:
FP8

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Inkling 2 Mini running on a single RTX 4090 at about 190-220 tokens/s with vLLM, up from roughly 117 out of the box. Setup uses NVFP4 experts (W4A16) plus FP8 W8A16 attention, head and embeddings, with FP8 KV cache, fitting in 24 GB at a 35k context. The run uses the model's built-in drafter and required small patches to vLLM, which does not yet support 4090s for this model. An initial quality check on sample prompts reported about a 0.4% loss in perplexity; the user plans further testing.

Nov 21, 2026
Tone: positiveAMD hardware
generation:
14-24 tokens/s
quant:
UD-IQ1_S (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Inkling-2-small running on a single AMD Strix Halo at 14-24 tok/s decode with drafting and over 310 tok/s prefill. Setup is libosmos v0.8.0 with a patch pack based on the Unsloth UD-IQ1_S quant, about 156GB. The pack reports a KLD of 0.196 and top-1 agreement of 85.8%, with more optimizations promised in later versions.

Nov 16, 2026
Engines people run Inkling-2 with
EngineReports
llama.cpp2
vLLM1

Get a weekly email of new Inkling-2 reports on any GPU.

Email me new reports

Reported configurations

Individual observations from this page, not expected speeds or a ranking. Select a model to open its report, with the source and offloading conditions. Reported context may be a limit rather than actual input length.

Model / hardwareQuant / engineReported contextGeneration
Inkling-2 Mini
AMD Radeon AI PRO R9700 32GB
Q8_0
llama.cpp
Not reported30.0 tokens/s