Use cases
codingtool-uselong-context
Summary
User publishes two EXL3 quants of Qwen3.8-Flash-Next for the sushi engine on Apple Silicon, reporting about 57 tok/s decode for the 4-bit pack and 63 tok/s for the 5-bit on an M4 Max with 128 GB.
Routed experts come from ExLlamaV3 v1.6.0; dense layers, embeddings, vision tower and MTP head are copied from beamster's Sushi-4bpw. Prefill at 64K is about 690 tok/s.
The 4-bit pack is 63.7 GiB and the 5-bit 77.8 GiB; KLD against BF16 is 0.94 for 4-bit and 0.79 for 5-bit. Needles in a 400K-token document scored 4/4 for both, and tool calling 28/28 for the 5-bit. The eval is 24 prompts on one machine.