Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports the AtomicChat Qwen3.8-Flash-Next GGUF quant on an M4 Max 128GB Mac Studio with cold-start prefill of about 500 t/s; no decode speed is given.
Setup is llama.cpp (qwen4exp branch) with the AtomicChat GGUF, whose PLE n-gram table is paged from its file via mmap, so the model takes about 65GB of RAM instead of 106GB.
For comparison, on oMLX offloading PLE to SSD slows prefill from 600 t/s to 180 t/s, and an open oMLX PR makes that cold prefill almost 3 times faster.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingvisionlong-context
User reports Qwen3.8-Flash-Next at ~60 tok/s serial decode and ~730 tok/s prefill on an M4 Max 128GB.
Setup is mlx-serve 26.8.11 with a mixed 4-bit/8-bit MLX pack, ~75 GB resident, prefix cache on, images included. With MTP enabled decode reaches 78 tok/s (+41% on code, a few percent slower on prose). A needle at 24.8k tokens was recovered with sparse attention engaged.
The pack includes the MTP head and vision tower; the 51B n-gram table ships as a 32 GB mmapped file that is not resident. The author recommends a re-quantized iQ-MLX-4.7bpw version instead.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagentic
User reports Qwen3.8-Flash-Next at 57.4 tok/s decode and 586 tok/s prefill on an M4 Max 128GB, where the model fits fully in memory.
Setup is oMLX with the Jundot oQ4e+MTP MLX checkpoint; the 36GB paging configuration uses oMLX 0.7.0 expert offload with resident fraction 0.28 (143/512 experts per layer), PLE on SSD and MTP off.
The same model on an M4 Max Studio 36GB measures 15.55 tok/s steady (n>=1024) and 13.0 tok/s short-run (n=128); exact replay on repeated prompts adds about 40% (14.3 to 20.1 tok/s). Other memory tiers are simulated: 48GB at 192 experts ~15.9 tok/s, 48GB at 128 experts ~11.8 tok/s, M4 Pro 32GB ~9.7 tok/s, M4 24GB ~4.9 tok/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codinglong-contextvision
User reports Qwen3.8 Flash-Next at 73.8 tok/s on an M4 Max with 128 GB unified memory.
Setup is mlx-serve with Q4 affine routed experts, BF16 core and verification head, native MTP at depth 4, and the full 262,144-token context; the 51.2 GB n-gram lookup table streams from SSD.
Matched warm workloads measured 71-74 tok/s for code and technical prose, with serial decode at 33.2 tok/s and a 2.22x MTP speedup. Long-context prefill reached 280.7 prompt tok/s at 248,445 tokens.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next at 50~70 tg/s at 128k context on an M4 Max 128GB.
Setup is oMLX 0.7.0 with an oQ4e MLX quant and MTP speculative decoding enabled; memory use is 75~80 GB at that context length.
User says older versions dropped from 60 tg/s to 35 tg/s once context passed 30k, and that the speed now holds as context grows.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B at 53.3 tok/s on a Mac Studio M4 Max 128GB.
Setup is oMLX 0.6.3rc2 with oQ4e 4-bit affine weights, 262,144 context, ANE prefill and native MTP speculative decoding with k=3 draft tokens, single stream.
Code generation measured 72.1 tok/s and prefill at 4K 273.7 tok/s. oMLX 0.6.1 with MTP gave 48.0 tok/s prose and 65.5 tok/s code; oMLX 0.6.3rc2 with ANE off gave 47.9 tok/s for both; vllm-metal 0.3.0 at 8-bit without MTP gave 13.2 tok/s code.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3-30B-A3B 4-bit at 127.7 tok/s on an M4 Max 128GB.
Setup is vllm-mlx, a native MLX inference server, with greedy decoding and single-stream decode.
Two other models were also measured on the same machine: Qwen3-0.6B 8-bit at 417.9 tok/s and Llama-3.2-3B-Instruct 4-bit at 205.6 tok/s. Continuous-batching results, KV-cache quantisation and MoE top-k sweeps are documented separately.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
coding
User benchmarks three Qwen3.8-Flash-Next REAP builds for Apple Silicon on an M4 Max 40-core GPU with 128 GB unified memory at 4K context.
Setup is oMLX with mixed-precision packages, 48 transformer layers, top-10 routing, a retained 512-expert native MTP predictor, and 262k configured context. The REAP-384 oQ5e build scored 97/100 HumanEval no-thinking and 61/100 LiveCodeBench no-thinking at 679.1 PP tok/s, 53.9 TG tok/s, and 72.8 GB peak memory.
The REAP-384 oQ4e build reached 689.5 PP tok/s and 59.3 TG tok/s at 61.5 GB peak, while REAP-288 oQ4e reached 692.3 PP tok/s and 48.7 TG tok/s at 46.0 GB peak. Accuracy figures are 100-question stratified samples and throughput is workload-specific.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports running a model in-browser with custom WebGPU kernels, at speed comparable to llama.cpp.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks Qwen3.6-35B-A3B on a Mac with OMLX, comparing MTP enabled against disabled.
MTP shows minimal speedup for the 35B MoE model but roughly 2x for the 27B dense model.
Results include pp and tg t/s at various context lengths and batch sizes.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
long-contextagentic
User benchmarks Qwen3.8 27B on an M4 Max 128GB across five runtime and quant stacks at contexts from 32K to 256K.
The stacks are oMLX AWQ 5-bit, oMLX oQ8e, mlx-dspark, and MTPLX 4-bit and 8-bit.
Best decode at 256K is oMLX AWQ 5-bit at 15.2 t/s, while MTPLX collapses at 256K. Prefix caching is crucial for effective speed.
User reports Qwen 3.8 27B as the first model to break 94% on the cupel benchmark.
Setup is llama.cpp with the unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS quant, which outperformed other 4-bit quants.
Qwen 3.8 27B outperformed in coding but lost in general knowledge to Gemma 31B and Qwen 3.6.