llamaperf

Qwen3.8 125B (6B active) Flash-Next

on M2 Max 96GB · oMLX · 32,768 ctx

Tone: negativeApple hardware
Sep 10, 2026
Throughput
18.3 t/s gen · 235.7 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
oQ4e (MLX)
KV cache
8.0
MTP (Multi-Token Prediction)
on
System RAM
96 GB

Summary

User reports slow performance with Qwen3.8-Flash-Next on an M2 Max 96GB using the oMLX engine. Benchmarks give 124.1 pp TPS and 20.4 tg TPS at pp1024/tg128, 170.7 pp TPS and 16.7 tg TPS at pp4096/tg128, 214.3 pp TPS and 18.5 tg TPS at pp8192/tg128, and 235.7 pp TPS and 18.3 tg TPS at pp16384/tg128. Setup is oMLX, a custom MLX-based engine, with 32768 context and a turboquant_kv_bits 8.0 KV cache. The user also tried oQ4e-fp16-mtp, oQ3-fp16-mtp and oQ3-MTP variants. The fp16 variants are even slower at roughly 150 pp TPS and 8 tg TPS. The user notes llama.cpp achieves 350-400 pp TPS. The model is MoE with 6B active parameters.