llamaperf

Qwen3.8 125B (6B active) Flash-Next

on M5 Pro 48GB · llama.cpp · 4,096 ctx

Tone: mixedApple hardware
Sep 26, 2026
Throughput
18-22 t/s gen
Quant
UD-IQ1_M (GGUF)
Flash Attention
on
System RAM
48 GB

Use cases

codingagentic

Summary

User reports Qwen3.8-Flash-Next at roughly 18-22 tok/s generation on an M5 Pro MacBook Pro with 48GB unified memory. Setup is llama.cpp b10930 with Unsloth UD-IQ1_M GGUF, 4096 context, flash attention on, 12 experts on CPU, and --load-mode none --lazy-mode on to avoid Metal OOM. At 16384 context for an OpenCode test, a 10,634-token prompt took 503 seconds of prefill then generated 192 tokens in 16 seconds; a follow-up reused 10,825 cached tokens but 5,113 more tokens took 165 seconds, and a 28,081-token request was rejected. About 94% of inference time went to prefill, and the user calls it not practical for agentic coding on this setup.