llamaperf

Qwen3.8 125B (6B active) Flash-Next

on M5 Pro 64GB · llama.cpp · 29,000 ctx

Tone: positiveApple hardware
Sep 18, 2026
Throughput
27.6 t/s gen · 367.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
Q4_0 (GGUF)
System RAM
64 GB

Use cases

coding

Summary

User reports Qwen3.8 Flash-Next at ~27.6 t/s generation and ~367 t/s prompt processing on an M5 Pro 64GB, with the 95.5 GiB model streamed from SSD. Setup is a llama.cpp fork with expert streaming, a draft head, depth 3 with a 0.3 confidence floor, sparse attention, and Metal MoE fusion, using a bartowski Q4_0 checkpoint with Q8_0 output layer and unsloth UD-IQ4_XS spliced in. Generation drops to 18-18.6 t/s with the draft head off and ~20.6 t/s at 29k context in real chat use. Paired 40-chunk perplexity improved from 5.2777 to 4.3148.