llamaperf

Qwen3.8 125B (6B active) Flash-Next

on M3 Ultra 512GB · llama.cpp · 32,768 ctx

Tone: negativeApple hardware
Sep 28, 2026
Throughput
25.7 t/s gen
Quant
Q8_0 (GGUF)
System RAM
512 GB

Summary

User reports Qwen3.8-Flash-Next at 25.7 t/s aggregate decode on an M3 Ultra 512GB, with throughput collapsing as concurrent long sequences are batched. Setup is llama.cpp (Unsloth prebuilt b10830) with Q8_0 GGUF, 32k context, flash attention on, and 8 slots. At 32k context, N=1 gives 25.7 t/s aggregate, N=2 gives 8.6 t/s, and N=4 gives 5.5 t/s. Two separate -np 1 processes reach 39.4 t/s aggregate, 4.6x faster than one process batching the same two sequences. User also sees the same cliff with GLM-5.3-Flash, going from 40 to 10.6 t/s aggregate at N=4 between 256 and 8k context, and uses two -np 1 processes behind a load balancer as a production workaround.