llamaperf

Qwen3.8 125B (6B active) Flash-Next

on M5 Ultra 256GB · MTPLX · 4,096 ctx

Apple hardware
Oct 9, 2026
Throughput
53.0 t/s gen · 2200.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
System RAM
256 GB

Summary

User reports Qwen3.8 Flash-Next at 53 t/s generation and 2,200 t/s prefill on a Mac Studio M5 Ultra with 256 GB unified memory. Setup is MTPLX 2.12.0 with speculative decoding and 55-65% draft acceptance; quantization and KV cache are unspecified. The 53 t/s figure is at roughly 4K context, with 51 t/s at roughly 16K context; the 2,200 t/s prefill is at a roughly 17K prompt. Resident memory was not measured; the 99 GB figure is the on-disk size only. Measurements are medians of three runs per cell, and the author notes small models served traffic on the same GPU throughout testing.