Qwen3.8 125B (6B active) Flash-Next
on M4 Max 128GB · llama.cpp
Summary
User reports the AtomicChat Qwen3.8-Flash-Next GGUF quant on an M4 Max 128GB Mac Studio with cold-start prefill of about 500 t/s; no decode speed is given.
Setup is llama.cpp (qwen4exp branch) with the AtomicChat GGUF, whose PLE n-gram table is paged from its file via mmap, so the model takes about 65GB of RAM instead of 106GB.
For comparison, on oMLX offloading PLE to SSD slows prefill from 600 t/s to 180 t/s, and an open oMLX PR makes that cold prefill almost 3 times faster.