Qwen3 8B
on M3 Pro 18GB · llama.cpp · 1,024 ctx
Use cases
summarizationtool-use
Summary
User benchmarks Qwen3-8B at 25.2 tok/s on an M3 Pro 18GB.
Setup is llama.cpp b11246 with Q4_K_M and 1024-token input, 256 output tokens, single request.
Runs were taken under memory pressure so absolute speeds are pessimistic; MLX-LM reaches 35.5 tok/s at 4 concurrent requests while llama.cpp plateaus at 21.0.