CodeLlama 70B base / Q5_K_M / M2 Ultra Metal b2050 / native 9-token prompt and 399 eval runs
on M2 Ultra 64GB · llama.cpp b2050 (19122117) / Metal · 512 ctx
Summary
User reports CodeLlama-70B at 8.7 t/s generation and 15.1 t/s prompt processing on a Mac Studio M2 Ultra with 64 GB unified memory.
Setup is llama.cpp build 2050 with Q5_K_M and F16 K/V cache, context and batch 512, 16 CPU threads, and 81/81 layers offloaded to the single Metal device.
The source prints 8.72 t/s generation over 399 evaluation runs and 15.06 t/s prompt processing over 9 tokens; the stored figures are rounded to 0.1 t/s.