llamaperf
Oct 14, 2026
Throughput
49.1 t/s gen
Quant
4-bit (MLX)
System RAM
96 GB

Summary

User reports 49.1 t/s generation. Source evidence (verbatim): Source: https://arxiv.org/html/2605.00519v1 # First-hand M3 Ultra 96GB Qwen3-Next 80B native MLX 4-bit generation measurement Original experimental paper Silicon Showdown, arXiv 2605.00519v1, May 2026. Selected Table 5 row: Apple M3 Ultra 96GB, 28 CPU cores, 60 GPU cores; installed unified memory 96GB. macOS 26 Tahoe, MLX v0.30.6. Qwen3-Next-80B-A3B native MLX 4-bit: generation 49.1 tokens/s, 0% CPU offload. Long prompt approximately 496 tokens; output 256 tokens, greedy decoding. Authors report arithmetic mean of three consecutive timed runs after two untimed warmups and fixed random seed. This is their published generation-phase measurement, not a speed derived from TTFT. Prompt-processing speed unreported; leave promptTps null. Context capacity, KV quantization, speculation, concurrency and resident allocation unreported. Verified base-model reference explicitly cited by the paper: https://huggingface.co/Qwen/Qwen3-Next-80B-A3B-Instruct . This is not the tested MLX quantization repository; exact quantized artifact and revision unknown. Do not infer disk size or resident memory. Table 2 names the 96GB machine, not the followed 256GB configuration. Other models, machines and Nvidia offload comparisons are separate runs. Source: https://arxiv.org/html/2605.00519v1 .