Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.
Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.
Free to use. No account needed. Memory estimates and community measurements are labelled separately.
Search for a model to see its reported speeds across GPUs and Macs.
These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →
Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.
M2 Max 96GB · llama.cpp
User reports a vision benchmark champion model at 70 s/img on an Apple M2 Max 96GB with llama.cpp b9690. The model gives the best quality and stability with thinking disabled, and completed 90/90 successful runs.
M2 Max 96GB · llama.cpp · 256,000 ctx
User benchmarks Qwen 3.6 27B F16 on an M2 Max 96GB with llama.cpp and MTP speculative decoding, reporting 8-18 tok/s depending on the task. Without MTP the same setup reaches 6.6 tok/s. A Q8 quant was also tested but performed worse. Context up to 150k+ tokens is usable. The user notes chat template fixes were critical, and the setup was used for agentic coding to create a Pacman game.
M2 Max 96GB · llama.cpp · 262,144 ctx
User reports MTP speculative decoding gives a 2.5x speedup on an M2 Max 96GB. Setup is Q5_K_M quant with a q4_0 KV cache. The user also provides hardware recommendations for various Apple Silicon and NVIDIA GPUs.
On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking