llamaperf

Qwen3.5 4B

on M4 Max 48GB · llama.cpp

Tone: positiveApple hardware
Sep 30, 2026
Throughput
85.0 t/s gen
Quant
Q4_K_M (GGUF)
System RAM
48 GB

Use cases

summarizationtool-use

Summary

User reports Qwen3.5 4B at ~85 tok/s decode on an M4 Max 48GB, running a Mac meeting notetaker stack. Setup is llama.cpp with Q4_K_M weights; the full on-device stack totals 6.8 GB across five models. A 26-minute meeting produced finished notes in 33 s warm. Speaker error dropped from 43.4% to 14.6% DER on AMI, and autocomplete p95 improved from 1.83 s to 0.49 s. MiniCPM5 2B was tried for notes but repeated itself and misassigned action items, so Qwen3.5 4B stayed the default.