Inkling-2 Mini
AMD Radeon AI PRO R9700 32GB · llama.cpp
- generation:
- 30.0 tokens/s
- quant:
- Q8_0 (GGUF)
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Inkling-2-Mini at about 30 tok/s on a single Radeon AI PRO R9700, which they call pretty slow. Setup is llama.cpp built from source with the Q8_0 GGUF from ggml-org; the model only runs on one card. The unsloth Q8_K_XL build did not work with the latest llama.cpp, and its Mainline_Support variant produced gibberish.