llamaperf

Qwen3.6 35B-A3B Q8_0 / Metal native PP4096 and separate TG128 / batch512

on M3 Ultra 512GB · llama.cpp b49650adb31f2e49a0d76113aeb1792134fd8413 / Metal

Apple hardware
Oct 14, 2026
Throughput
81.2 t/s gen · 2324.6 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
Q8_0 (GGUF)
KV cache
F16
Flash Attention
on
System RAM
512 GB

Summary

User reports Qwen3.6-35B-A3B Q8_0 at 81.2 t/s generation and 2324.6 t/s prompt processing on a Mac Studio with an Apple M3 Ultra and 512 GiB unified memory. Setup is llama.cpp with Metal GPU offload, Flash Attention on, F16 KV cache, 8 threads, and batch/ubatch 512/512; the Mac-only control arm uses one GPU. PP4096 and TG128 are separate synthetic tests, with TG128 starting from an empty context rather than generation after prefill. The article also reports a continuous 4096-input plus 128-output workload at 3.409 seconds, and a separate tuning pass at the same batch/ubatch gives PP4096 2323.37 and TG128 80.68 t/s.