Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.
Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.
Free to use. No account needed. Memory estimates and community measurements are labelled separately.
Search for a model to see its reported speeds across GPUs and Macs.
These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →
Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.
AMD Radeon Pro W7900 · llama.cpp · 128,000 ctx
User reports LFM2.5-24B-A2B at ~95 t/s sustained on an AMD Radeon Pro W7900. Setup is llama.cpp with ROCm, Q8 quant, full 128k context, taking 30GB VRAM. User swapped it in for Ornith 1.5-35B in Pi Agent workflows and says it holds up well for mechanical and long-running recursive tasks; insight testing in council mode is pending.
NVIDIA RTX 5090 · llama.cpp
User reports LFM2.5-8B-A1B at 563.12 t/s generation and 30,680.66 t/s prompt processing on an RTX 5090. Setup is llama.cpp llama-bench (build b9653) with Q8_0 weights, all layers on GPU (ngl=99), single request, no speculative decoding. The dataset card also lists llama-bench rows for gpt-oss-20b, Nemotron-3-Nano-30B-A3B, Qwen3.6-35B-A3B, Qwen3-Coder-Next, Gemma 4 12B, Qwen3.6-27B, Gemma 4 31B, and Ling-3.0-flash, plus rows for Qwen3.6-27B-MTP, Qwen3.6-27B Q6_K, gpt-oss-120B, Mistral Small 4 119B, and several quant ladders; some runs used vLLM and SGLang.
M1 8GB · llama.cpp · 32,000 ctx
User compares LFM2.5 2.6B and MiniCPM5 2B on small agentic tool-use tasks on an M1 Air. LFM2.5 2.6B runs at 200 t/s prompt and 22 t/s generation with llama.cpp, QAD-Q4_0 GGUF, 32k context, using about 2.5 GB of RAM. MiniCPM5 2B runs at 162 t/s prompt and 16 t/s generation with a DSpark draft model, Q8_0 GGUF, 32k context, q5_1 KV cache, using about 3.8 GB of RAM. User calls LFM2.5 the clear winner: faster, less RAM despite more parameters, and better aligned for English conversation. MiniCPM5 is noted as often responding in Chinese and hallucinating less, but LFM2.5 tends to hallucinate as conversations get longer.
NVIDIA RTX 4050 6GB · LM Studio · 256,000 ctx
User benchmarks 20 small models on an RTX 4050 6GB. Models tested include Granite 4.1 3B, Gemma-4-agentic-e2b, Nemotron-3-Nano-4B and LFM2.5-8B-A1B. The user selects LFM2.5-1.2B-Instruct as a cheap always-on model.
AMD Strix Halo 128GB · llama.cpp · 128,000 ctx
User reports Liquid AI's LFM2.5-2.6B at 113 tok/s on a Ryzen AI Max+ 395, a CPU speed from the vendor's own benchmarks. Setup is the official Q4_K_M GGUF of around 1.67 GB, a 2.69B parameter model with 128K context, using under 2.5 GB memory in the vendor's tests. The vendor also reports 30 tok/s on a phone and 220 tok/s on an M5 Max, and the user notes independent results are needed. Benchmarks are ToolSandbox 77.83, IFBench 59.17, BFCLv4 56.88 and LiveCodeBench 59.41, and the model card does not recommend it for agentic coding.
On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking