Inkling-2 Mini
AMD Radeon AI PRO R9700 32GB · llama.cpp
- generation:
- 30.0 tokens/s
- quant:
- Q8_0 (GGUF)
User reports Inkling-2-Mini at about 30 tok/s on a single Radeon AI PRO R9700, which they call pretty slow. Setup is llama.cpp built from source with the Q8_0 GGUF from ggml-org; the model only runs on one card. The unsloth Q8_K_XL build did not work with the latest llama.cpp, and its Mainline_Support variant produced gibberish.