llamaperf

Qwen3.8 125B (6B active) Flash-Next

on M5 Ultra 96GB · Inco Splash · 1,024 ctx

Apple hardware
Oct 29, 2026
Throughput
145.6 t/s gen · 3880.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
UD-IQ4_XS (GGUF)
System RAM
96 GB

Use cases

tool-use

Summary

User benchmarks Qwen3.8 Flash-Next at 145.6 t/s decode and 3,880 t/s prefill at 1k context on an M5 Ultra 96GB, using Inco Splash 1.5.0 with the UD-IQ4_XS GGUF and an auto-paired DFlash2 drafter. Setup streams weights from disk at about 78 GB resident with 262k context; the 88 GB GGUF sits in the HF cache. Decode falls to 115.4 t/s at 4k and 112.2 t/s at 16k, then reads 163.0 t/s at 32k, 139.8 t/s at 64k and 135.6 t/s at 128k. Figures are medians of 3 reps with unique-nonce prompts and one request in flight; the machine was also serving an agent chat throughout. The post compares Splash against Sushi 2.75bpw, whose numbers are not given here.