Qwen3.8 125B (6B active) Flash-Next
on M5 Pro 64GB · llama.cpp · 29,000 ctx
Use cases
coding
Summary
User reports Qwen3.8 Flash-Next at ~27.6 t/s generation and ~367 t/s prompt processing on an M5 Pro 64GB, with the 95.5 GiB model streamed from SSD.
Setup is a llama.cpp fork with expert streaming, a draft head, depth 3 with a 0.3 confidence floor, sparse attention, and Metal MoE fusion, using a bartowski Q4_0 checkpoint with Q8_0 output layer and unsloth UD-IQ4_XS spliced in.
Generation drops to 18-18.6 t/s with the draft head off and ~20.6 t/s at 29k context in real chat use. Paired 40-chunk perplexity improved from 5.2777 to 4.3148.