llamaperf

Qwen3.8 125B (6B active) Flash-Next

on M5 Ultra 256GB · llama.cpp · 250,000 ctx

Tone: mixedApple hardware
Oct 3, 2026
Throughput
17.0 t/s gen · 1069.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
Q8_0 (GGUF)
System RAM
256 GB

Use cases

agenticcodinglong-context

Summary

User benchmarks Qwen3.8-Flash-Next 8-bit on a Mac Studio M5 Ultra 256GB, reporting 17 t/s decode and 1,069 t/s prefill at 250k context on llama.cpp (Metal). Setup is llama.cpp with Q8_0 weights and MTP off; oMLX 0.7.0rc1 reaches 51 t/s decode and 1,365 t/s prefill at the same depth, and 88 t/s decode at 10k with MTP on. A cold 250k-token prompt took 234 s before the first token, while the cached next step took 0.64 s. DeepSeek-V4-Flash at 400k and 500k context took 934 s and 1,311 s cold. The user notes one overnight agent run per setup and that the DeepSeek engine comparison mixes two model variants.