llamaperf

Qwen3.8 125B (6B active) Flash-Next

on M5 Max 128GB · llama.cpp · 358,400 ctx

Tone: mixedApple hardware
Sep 7, 2026
Throughput
11.5 t/s gen · 333.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
UD-Q2_K_XL (GGUF)
KV cache
F16
System RAM
128 GB

Use cases

long-context

Summary

User reports Qwen3.8-Flash-Next at 79 GB in 2-bit running at 350K context for 3.5 hours on a 128 GB M5 Max, over 100 turns. Prefill runs 1,561 t/s at 5.6K context tapering to 318 t/s at 111K, with normal turns at 77-854 t/s from prefix reuse. Decode is about 30-35 t/s at small context down to 11.5 t/s at 169K, with a dip to 7.7 t/s at around 140K due to Low Power Mode. Decode figures are effective throughput with speculative decoding. The user calls the experience strong for the first ~100K context, then reports role confusion on long-tail tasks, suspecting the 2-bit quant or preview-model long-context quality.