llamaperf

Qwen3.8 125B (6B active) Flash-Next

on M4 Max 128GB · sushi

Tone: positiveApple hardware
Oct 17, 2026
Throughput
57.0 t/s gen
Quant
4bpw-sh (EXL3)
System RAM
128 GB

Use cases

codingtool-uselong-context

Summary

User publishes two EXL3 quants of Qwen3.8-Flash-Next for the sushi engine on Apple Silicon, reporting about 57 tok/s decode for the 4-bit pack and 63 tok/s for the 5-bit on an M4 Max with 128 GB. Routed experts come from ExLlamaV3 v1.6.0; dense layers, embeddings, vision tower and MTP head are copied from beamster's Sushi-4bpw. Prefill at 64K is about 690 tok/s. The 4-bit pack is 63.7 GiB and the 5-bit 77.8 GiB; KLD against BF16 is 0.94 for 4-bit and 0.79 for 5-bit. Needles in a 400K-token document scored 4/4 for both, and tool calling 28/28 for the 5-bit. The eval is 24 prompts on one machine.