Mellum2.1
on Unknown GPU · llama.cpp · 48,000 ctx
Use cases
coding
Summary
User reports Mellum 2.1 Q4 with MTP on llama.cpp (Vulkan) giving around 25ish tok/s prompt processing and 12ish tok/s generation at a capped 48k context.
Setup is CPU-only on an i7 1365U with 32GB DDR5 RAM on Ubuntu, run through the Pi harness with no extensions.
Both figures are approximate ranges, so no single throughput is recorded. User asks how Gemma 4 26B-A4B QAT with MTP, Ornith 1.5 9B and Spark X2.5 4B would compare for Python and shell scripting, and whether Pi can be used as a coding sub-agent inside GitHub Copilot without MCP.