llamaperf

Qwen3.8 125B (6B active) Flash-Next

on M5 Ultra 256GB · oMLX · 1,024 ctx

Apple hardware
Oct 11, 2026
Throughput
135.2 t/s gen · 2458.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
oQ6e (MLX)
KV cache
8-bit
System RAM
256 GB

Summary

User benchmarks Qwen3.8-Flash-Next oQ6e at 135.2 t/s decode and 2,458 t/s prefill on an M5 Ultra 256GB, comparing oMLX 0.7.0 and 0.7.1.dev1. Setup is oMLX with MLX oQ6e weights, 8-bit KV cache, Lightning MTP on, 1K context, single request. Across 4K-200K contexts, average decode is 120.9 t/s and prefill 4,761 t/s; 2 parallel requests give 74.2 t/s per request and 4 parallel 41.0 t/s per request. GLM-5.3-Flash oQ4 reaches 97.9 t/s decode and 2,264 t/s prefill, and Flash-Next oQ4e 121.6 t/s decode and 5,450 t/s prefill.