Qwen3.8 125B (6B active) Flash-Next
on M4 Pro 24GB · 65,536 ctx
Use cases
codinglong-context
Summary
User reports Qwen3.8-Flash-Next 125B at 11 t/s generation and 180 t/s prefill on a Mac mini M4 Pro with 24GB RAM, using a custom inference implementation.
Setup uses ISTA Lab's GSQ RCO IQ3 XXS quantization, roughly 76GB on disk, with support for up to 65K tokens of context. The model streams from storage, with peak runtime memory measured at 9.5GB.
Speed is 11 t/s with short prompts and 8.5 t/s with longer contexts; the first implementation ran under 3 t/s generation and around 40 t/s prefill. The user is also testing it with a coding agent starting from over 10k tokens of context.