Llama 3.2 3B Impish

User reports a 3B roleplay finetune running fully on an iPhone at around 6 tok/s on an A18 chip. Setup is llama.cpp with Metal, Impish Llama 3B (a Llama 3.2 3B RP finetune) with a rank 16 LoRA fused and requantized to Q4_K_M, KV cache at q8_0, all layers offloaded, about 2.0 GB on disk. The user notes the speed is from an early measurement and has not been re-measured on the current build. Context is 4096 on 4 GB phones, 6144 on 6 GB phones, and 8192 on 8 GB phones. The user reports that scaling the history trim budget to about 0.68 x n_ctx raised long-conversation fact recall from 25% to 75%, that BM25 retrieval re-injected superseded facts (+20.8pp) until filtered by rare entities (-18.8pp, CI [-35.4, -6.2]), and that scene directions placed as their own block after the final user turn landed 16/16 versus 0/16 in the reminder slot.