Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.
Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.
Free to use. No account needed. Memory estimates and community measurements are labelled separately.
Search for a model to see its reported speeds across GPUs and Macs.
These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →
Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.
M5 Pro 48GB · Ollama
User reports the Flint model at around 112 tok/s on an M5 Pro 48GB Mac. Setup is Ollama, with the speed read from Ollama's own eval_count / eval_duration and the model unloaded after each run. The figure comes from the author's own speed-test tool, which streams the answer and counts reasoning tokens toward the speed.
M5 Pro 48GB · Global Agent
User reports Qwen3-Coder-Next 80B-A3B at 34.5 tok/s decode on an M5 Pro 48GB Mac, running a full agentic coding task in 44s. Setup is the Global Agent engine (MLX-based) with 6-bit weights and expert streaming from SSD, speed mode, peak memory 31.5GB. The post also benchmarks Qwen3.8-Flash-Next 229B-A7B (19.2 tok/s), Qwen3-235B-A22B (9.19 tok/s) and MiniMax-M3 426B-A24B (4.62 tok/s) on the same machine, plus 16GB and 32GB memory-capped runs; the 16GB Qwen3-Coder-Next run reached 41.2 tok/s. Exact mode figures are lower (17.7, 7.88, 2.62, 2.20 tok/s).
M5 Pro 48GB · mlx-lm · 4,096 ctx
User reports Qwen3.5-27B affine 4bit prefilling a 4096-token prompt at 461 tok/s on an M5 Pro 48GB (20-core GPU) with stock mlx 0.32.0 and mlx_lm 0.31.3 in a clean venv. The post is a bug report against oMLX 0.5.0, whose bundled MLX build fails a NAX metallib scan, so its M5 dispatch never engages: the same model prefilled at only 180 tok/s at 4096 tokens there, and 262 tok/s with OMLX_NAX=1. No decode figure is given beyond saying decode is normal. Qwen3.5-35B-A3B 4bit prefilled at 1,798 tok/s in the clean venv against about 900 on oMLX 0.5.0.
M5 Pro 48GB · Ollama · 16,384 ctx
User compares DeepSeek V4 Flash 9B distill against its base model Qwen3.5 9B on a MacBook Pro M5 Pro with 48 GB RAM. Both models gave the same answers on 6/8 tasks, but the distill used fewer tokens (5480 vs 8975). Throughput was near identical (44 vs 41 tok/s). The distill was slower on open-ended tasks (1467 tokens vs 867 for a buffer overflow explanation). The base model failed arithmetic due to a token cap.
M5 Pro 48GB · llama.cpp
User compares MLX and GGUF for Qwen3.8 27B on an M5 Pro, reporting generation at about 19 t/s. Setup is llama.cpp with Metal, which the user says now matches MLX prefill at roughly 300-350 t/s. The user questions whether MLX is still needed and mentions MTP and oMLX.
On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking