llamaperf
Sep 10, 2026
Throughput
22.0 t/s gen · 261.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
ternary
System RAM
12 GB
VRAM reported
12 GB

Use cases

codingagentic

Summary

User reports Millie, a series of compressed agentic models derived from Agents-A1 (a Qwen 3.5 35B-A3B finetune), running on an iPhone 17 Pro with 12 GB RAM. The ternary expert model scores 56% on SWE-bench Verified at 22 t/s decode and 261 t/s pre-fill. The 2-bit expert version scores 60%. The software targets Macs with 16 GB+ memory and Linux gaming PCs with 16 GB+ system RAM and as little as 4 GB VRAM, with a harness forked from OpenAI Codex. The user is seeking testers for AMD GPUs and small 4-8 GB cards.