llamaperf

Qwen3.8 125B (6B active) Flash-Next

on M4 Pro 24GB · 65,536 ctx

Tone: positiveApple hardware
Oct 28, 2026
Throughput
11.0 t/s gen · 180.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
GSQ RCO IQ3 XXS
System RAM
24 GB

Use cases

codinglong-context

Summary

User reports Qwen3.8-Flash-Next 125B at 11 t/s generation and 180 t/s prefill on a Mac mini M4 Pro with 24GB RAM, using a custom inference implementation. Setup uses ISTA Lab's GSQ RCO IQ3 XXS quantization, roughly 76GB on disk, with support for up to 65K tokens of context. The model streams from storage, with peak runtime memory measured at 9.5GB. Speed is 11 t/s with short prompts and 8.5 t/s with longer contexts; the first implementation ran under 3 t/s generation and around 40 t/s prefill. The user is also testing it with a coding agent starting from over 10k tokens of context.