Qwen3.8 125B (6B active) Flash-Next
on M5 Max · llama.cpp
Summary
User benchmarks Qwen3.8-Flash-Next with DFlash2 speculative decoding on an M5 Max in llama.cpp v0.8.0, reporting a geometric mean of 87.9 t/s versus 75.1 t/s with MTP, a 17% gain.
Setup uses Q4_K_XL weights with a Q4_0 draft head; DFlash2 ran with n-max 5 and p-min 0.75, MTP with n-max 3 and p-min 0.75.
Per-prompt results: code 95.1 t/s, list 127 t/s, prose 51.9 t/s, 6k context 95.3 t/s. At 25K context DFlash2 roughly equals MTP. The Q4_0 draft head is about 350MB versus 2.2GB for the MTP head, though the compute buffer offsets the VRAM saving. Vision prompts work with DFlash2.