llamaperf

Qwen3.8 125B (6B active) Flash-Next

on M4 Max 128GB · llama.cpp

Tone: positiveApple hardware
Oct 8, 2026
Throughput
500.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
System RAM
128 GB

Summary

User reports the AtomicChat Qwen3.8-Flash-Next GGUF quant on an M4 Max 128GB Mac Studio with cold-start prefill of about 500 t/s; no decode speed is given. Setup is llama.cpp (qwen4exp branch) with the AtomicChat GGUF, whose PLE n-gram table is paged from its file via mmap, so the model takes about 65GB of RAM instead of 106GB. For comparison, on oMLX offloading PLE to SSD slows prefill from 600 t/s to 180 t/s, and an open oMLX PR makes that cold prefill almost 3 times faster.