Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.
Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.
Free to use. No account needed. Memory estimates and community measurements are labelled separately.
Search for a model to see its reported speeds across GPUs and Macs.
These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →
Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.
User reports Ornith 1.5 35B-A3B at 108 tok/s on an M2 Max 32GB, versus 66 tok/s with the macOS Aerial lock-screen animation playing. Setup is oMLX with a 4-bit MoE model, medians of 4-6 local runs over 512 output tokens. After an 8k-token prompt, generation was 89 vs 66 tok/s, and WindowServer CPU use rose from roughly 25% to 44% while the animation ran. Using caffeinate -i instead of -d let the display sleep and restored speed.
M2 Max 32GB · llama.cpp · 4,096 ctx
User benchmarks Qwen3.6-35B-A3B at 58.56 t/s generation and 671.40 t/s prefill on an M2 Max 32GB. Setup is llama.cpp build c1d0e7a00 with Metal backend, Q4_K_M GGUF (20.22 GiB), 4096-token prompt, no speculative decoding. The dataset reports mostly negative results: MoE beats dense Qwen3.8-27B by 5.4x generation, aggressive quantization bought little, MTP and DFlash2 speculative decoding were net losses on Metal, and q8_0 KV cache was free. MLX reached 86.9 t/s generation but only 296 t/s prefill versus llama.cpp's 57.8 t/s and 685 t/s. oMLX with paged SSD KV cache reached 13.7 t/s generation and 84 t/s prefill on the dense 27B.
M2 Max 32GB · 60,000 ctx
User reports Qwen3.6 35B A3B at 15-20 t/s on an M2 Max 32GB, with prompt processing taking 20 minutes for 40k tokens when the model fits in RAM. Setup is Q4_XS GGUF at 60,000 context with a Q8 KV cache. With SSD streaming the model runs at 4-10 t/s and prompt processing takes 50-60 minutes for 40-60k tokens. User asks whether to upgrade to an M1 Max 64GB, M3 Max 128GB, or a 24GB GPU build with 128GB RAM.
On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking