Qwen3.5 4B
on M4 Max 48GB · llama.cpp
Use cases
summarizationtool-use
Summary
User reports Qwen3.5 4B at ~85 tok/s decode on an M4 Max 48GB, running a Mac meeting notetaker stack.
Setup is llama.cpp with Q4_K_M weights; the full on-device stack totals 6.8 GB across five models.
A 26-minute meeting produced finished notes in 33 s warm. Speaker error dropped from 43.4% to 14.6% DER on AMI, and autocomplete p95 improved from 1.83 s to 0.49 s. MiniCPM5 2B was tried for notes but repeated itself and misassigned action items, so Qwen3.5 4B stayed the default.