Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.
Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.
Free to use. No account needed. Memory estimates and community measurements are labelled separately.
Search for a model to see its reported speeds across GPUs and Macs.
These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →
Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.
User reports Qwen3.8 Flash-Next 125B at ~18 tok/s on an M1 Max 64GB with the Espetero server, an inference server for Apple silicon written in C++20 and Metal. Setup streams experts from the SSD; the server loads MLX and GGUF files and serves OpenAI, Anthropic and Ollama APIs. DeepSeek-V4.1-Flash 552B runs at ~10 tok/s on a 128 GB Mac Studio, described as experimental.
M1 Max 64GB · LM Studio · 65,536 ctx
User benchmarks 48 local models as agents on an M1 Max 64GB, with qwen3.8-27b Q4_K_M reaching 16.9 tok/s at 64k context. Setup is LM Studio with GGUF weights, 15.7 GB of memory at 64k context, and 16 multi-step tool tasks plus 10 real-world failure cases run 3 times each. The model scored 99% quality, 100% agent and 98% on the real cases with 0 serious slips in 30 runs at 51 s per task. Ornith-1.5-35B-A3B is the faster pick at 52.4 tok/s with 94% quality and 98% agent. Prompt cache made a follow-up turn on the same long agent prefix 152x faster than cold start on GGUF models (median of 41).
M1 Max 64GB · MTPLX · 128,000 ctx
User benchmarks Qwen3.6-35B-A3B on an M1 Max 64GB, reporting 464 tok/s prefill and 62 tok/s decode at a 128K prompt with the MTPLX M1 fork. Setup is the MTPLX M1 fork v2.12.2-m1.11 with mlx-community 4-bit group-64 weights plus an MTP head, 8-bit paged affine KV cache, and speculative decoding on. Splash-M1 1.3.2-m1.1 reached 302 tok/s prefill and 41 tok/s decode at 128K, and oMLX 385 tok/s prefill and 33 tok/s decode. The fork was faster from 8K up on agent logs, 1.46x prefill and 1.5x decode at 128K, while Splash-M1 used less memory (23.5 GB vs 28.1 GB at 128K). The same post also compares Qwen3.8-27B, where the fork reached 132 tok/s prefill and 25.0 tok/s decode at 128K. Only the 2K, 32K and 128K agent-log cells were run three times; the rest are single runs.
M1 Max 64GB · LM Studio
User reports about 45 tokens/sec on a 7B Llama model in MLX on an M1 Max 64GB, but finds 7B too limited for most tasks. Setup is LM Studio; the user also tried a 32B Qwen model there and hit errors. User asks which models suit offline research, legal and medical analysis, light coding, CAD/PCB, Blender and image generation, and is looking to move up to 32B, 70B and 120B models.
User reports Qwen3.8-Flash-Next at 121 prompt t/s and 12 generation t/s on an M1 Max 64GB. Setup is MLX with oQ4e quant, 6-bit KV cache, 50% expert offload and MTP on. A later fp16 build of the same model gave OOM errors and, at 20% expert offload, only 90 prompt t/s and 6 generation t/s; the user asks whether the model or the settings are at fault.
M1 Max 64GB · LM Studio
User reports 33 t/s generation. Source evidence (verbatim): Source: https://famstack.dev/guides/mlx-vs-gguf-part-2-isolating-variables/ # Mac Studio M1Max64GB native generation after Gemma3QAT MLXdtype conversion Original author article March20,2026. Measured Mac Studio AppleM1 Max 64GB unified RAM,24GPUcores; CPUcorecount unknown. Gemma3-12B-it QAT4-bit MLX in LMStudio: author states generation increases from 28 to 33 tok/s after converting nonquantized dtype frombf16 tofloat16. No averaging or calculatedspeed used. Tested sourcequant repository verified https://huggingface.co/mlx-community/gemma-3-12b-it-qat-4bit ; convertedartifact generated locally with mlx_lm.convert --dtype float16, not a separate verifiedHFartifact. Testedrevision/runtimeversion/prompt/outputcounts/power/KVquant/speculation/residentRAM unknown. Separate reportedprefilltimes at655tokens6.2s→3.7s;8K114.4s→68.9s; do not derive nativeprefillT/s from approximate8K. Other charts report effectiveoutputtokens/totalrequesttime includingprefill, not generation; keep separate. Source: https://famstack.dev/guides/mlx-vs-gguf-part-2-isolating-variables/ . Rawlogs linkGitHub and were not fetched under sourceaccessstop. CurrentHFcard contains unrelated-looking multimodalmetadata; no testedhistoricalrevision established, so preserve artifact caveat. Selected run: locally converted float16 nonquantized dtype artifact, native generation 33 tok/s. Baseline before conversion is a separate run at 28 tok/s. Prompt processing speed unknown; do not derive it from prefill times.
M1 Max 64GB · llama.cpp b2296 (d5ab2975) / Metal
User reports Llama 2 7B Chat at 53 t/s generation and 499.1 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with IQ1_S quantization and Metal backend at ngl 99, one participating GPU. The figures are separate synthetic llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.
M1 Max 64GB · llama.cpp b2296 (d5ab2975) / Metal
User reports Llama 2 7B Chat at 50.5 t/s generation and 462.1 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with IQ2_XXS and Metal, ngl 99, on one GPU. The figures are separate synthetic llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.
M1 Max 64GB · llama.cpp b2296 (d5ab2975) / Metal
User reports Llama 2 7B Chat at 48.4 t/s generation and 472.2 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with IQ2_XS weights and Metal, ngl 99, on one GPU. The figures are separate llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.
M1 Max 64GB · llama.cpp b2296 (d5ab2975) / Metal
User reports Llama 2 7B Chat at 47.3 t/s generation and 469.5 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with IQ2_S quant, Metal backend and ngl 99, one participating GPU. The figures are separate synthetic llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.
M1 Max 64GB · llama.cpp b2296 (d5ab2975) / Metal
User reports Llama 2 7B Chat at 62.5 t/s generation and 512.2 t/s prompt processing on an M1 Max Mac Studio with 64GB unified memory. Setup is llama.cpp b2296 with Q2_K_S and Metal, ngl 99, on one GPU. The figures are separate llama-bench phases, PP512 and TG128, not one chat request; the source rates are 512.21 and 62.49 t/s.
M1 Max 64GB · llama.cpp b2296 (d5ab2975) / Metal
User reports Llama 2 7B Chat at 38.4 t/s generation and 462.4 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with IQ2_M quant, Metal backend and ngl 99, one participating GPU. The figures are separate synthetic llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.
M1 Max 64GB · llama.cpp b2296 (d5ab2975) / Metal
User reports Llama 2 7B Chat at 51.6 t/s generation and 493.4 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with Q2_K and Metal, ngl 99, on one participating GPU. The figures are separate synthetic llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.
M1 Max 64GB · llama.cpp b2296 (d5ab2975) / Metal
User reports Llama 2 7B Chat at 46.4 t/s generation and 470 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with IQ3_XXS and Metal, ngl 99, on one participating GPU. The figures are separate synthetic llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.
M1 Max 64GB · llama.cpp b2296 (d5ab2975) / Metal
User reports Llama 2 7B Chat at 47.9 t/s generation and 464.9 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with the IQ3_XS quant, Metal backend and ngl 99, using the author's locally converted weights. The figures are separate synthetic llama-bench phases, PP512 and TG128, not one 512-input/128-output request; TTFT was not measured.
M1 Max 64GB · llama.cpp b2296 (d5ab2975) / Metal
User reports Llama 2 7B Chat at 42.6 t/s generation and 466.8 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with Q3_K_S and Metal, ngl 99, on one GPU. The figures are separate llama-bench phases, PP512 and TG128, not one 512-input/128-output request; TTFT was not measured.
M1 Max 64GB · llama.cpp b2296 (d5ab2975) / Metal
User reports Llama 2 7B Chat at 47.7 t/s generation and 456.1 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with IQ3_S quant, Metal backend and ngl 99, one participating GPU. The figures are separate synthetic llama-bench phases, PP512 and TG128, not one 512-input/128-output request; TTFT was not measured.
M1 Max 64GB · llama.cpp b2296 (d5ab2975) / Metal
User reports Llama 2 7B Chat at 47.7 t/s generation and 459.5 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with IQ3_M and Metal, ngl 99, on one GPU. The figures are separate llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.
M1 Max 64GB · llama.cpp b2296 (d5ab2975) / Metal
User reports Llama 2 7B Chat at 44.2 t/s generation and 469.6 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with Q3_K_M and Metal, ngl 99, on one GPU. The figures are separate synthetic llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.
M1 Max 64GB · llama.cpp b2296 (d5ab2975) / Metal
User reports Llama 2 7B Chat at 40.9 t/s generation and 449.5 t/s prompt processing on an M1 Max Mac Studio with 64 GB unified memory. Setup is llama.cpp b2296 with Q3_K_L and Metal, ngl 99, on one participating GPU. The figures are separate synthetic llama-bench phases, PP512 and TG128, not one chat request; TTFT was not measured.
On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking