Bonsai 2 27B
on Unknown GPU · llama.cpp · 200,000 ctx
Summary
User asks for help running Bonsai 2 27B with Claude via a forked llama.cpp from Prism-ML, reporting a Jinja template error ("System message must be at the beginning") when using Claude, while the DeepSeek harness works fine.
Setup is llama.cpp (llama-server.exe) with a PQ2_0 GGUF quant, Q8_0 mmproj, q4_0 KV cache, 200000 context, -ngl 99, flash attention on, and 24 threads.
The log shows a prompt eval of 171.77 t/s over 385 tokens and generation of 26.23 t/s over 200 tokens; the user did not state the GPU.