M5 24GB

generation:

49.0 tokens/s

quant:

Q2_0

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next at 49 t/s decode on an M5 24GB MacBook Air. Setup is Splash 1.4.0 with a small fork, ISTA GSQ-RCO Q2_0 weights and a Flash-Next-DFlash2 draft model at 256K context. About 28% of expert slots (6,800 of 24,576, ~8.8 GB) stay resident and the rest are streamed from the SSD. Cold prefill without prompt cache is 522 t/s on a 32,620-token prompt and 485 t/s on 63,201 tokens. Real agent use in pi averages ~45 t/s with no prompt-cache losses, and draft acceptance is ~.70 for code and ~.80 for tool calls.

generation:

2.9 tokens/s

quant:

4bit (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Inkling-Small 276B-A12B at 2.86 tok/s decode on an M5 with 24GB, with a peak footprint of about 9.5 GB. Setup is Mference with the pipenetwork/Inkling-Small-MLX-4bit conversion: 3.4 GB resident and about 148 GB on disk. Decode holds at 2.93 tok/s on a 421-token prompt and 2.56 tok/s on a 2,785-token one, but that longer prefill took 535.9 s. The user also tested an M3 Ultra 256GB at 5.31-6.92 tok/s.