llamaperf
Oct 14, 2026
Throughput
12.0 t/s gen · 121.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
oQ4e (MLX)
KV cache
6bit
System RAM
64 GB

Summary

User reports Qwen3.8-Flash-Next at 121 prompt t/s and 12 generation t/s on an M1 Max 64GB. Setup is MLX with oQ4e quant, 6-bit KV cache, 50% expert offload and MTP on. A later fp16 build of the same model gave OOM errors and, at 20% expert offload, only 90 prompt t/s and 6 generation t/s; the user asks whether the model or the settings are at fault.