Qwen3 8B
M3 Pro 18GB · llama.cpp · 1,024 ctx
- generation:
- 25.2 tokens/s
- quant:
- Q4_K_M (GGUF)
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User benchmarks Qwen3-8B at 25.2 tok/s on an M3 Pro 18GB. Setup is llama.cpp b11246 with Q4_K_M and 1024-token input, 256 output tokens, single request. Runs were taken under memory pressure so absolute speeds are pessimistic; MLX-LM reaches 35.5 tok/s at 4 concurrent requests while llama.cpp plateaus at 21.0.