Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.
Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.
Free to use. No account needed. Memory estimates and community measurements are labelled separately.
Search for a model to see its reported speeds across GPUs and Macs.
These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →
Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.
Unknown GPU
User reports Liquid's latest model (LFM2.5) at 25 t/s sustained decode and 28 t/s sustained prefill at long contexts on an Intel 12th gen i5 with 24GB of mismatched DDR4 system RAM. The run was CPU only; no engine, quant or context length is stated. User calls it solid for spec driven development and asks why it gets no attention as an edge-deployment breakthrough.
AMD Radeon Pro W7900 · llama.cpp · 128,000 ctx
User reports LFM2.5-24B-A2B at ~95 t/s sustained on an AMD Radeon Pro W7900. Setup is llama.cpp with ROCm, Q8 quant, full 128k context, taking 30GB VRAM. User swapped it in for Ornith 1.5-35B in Pi Agent workflows and says it holds up well for mechanical and long-running recursive tasks; insight testing in council mode is pending.
M5 Max · 131,072 ctx
User reports LFM2.5-24B-A2B at 184 tok/s on an Apple M5 Max. Setup is GGUF weights with a 131,072-token context length. The figure is the vendor's on-device claim; the same release is quoted at up to 21K output tok/s on a single NVIDIA B300 GPU. The model is a 24B-parameter MoE with 2.45B active parameters, 64 experts per block with top-4 routing, and 36 trillion training tokens.
NVIDIA RTX 5090 · llama.cpp
User reports LFM2.5-8B-A1B at 563.12 t/s generation and 30,680.66 t/s prompt processing on an RTX 5090. Setup is llama.cpp llama-bench (build b9653) with Q8_0 weights, all layers on GPU (ngl=99), single request, no speculative decoding. The dataset card also lists llama-bench rows for gpt-oss-20b, Nemotron-3-Nano-30B-A3B, Qwen3.6-35B-A3B, Qwen3-Coder-Next, Gemma 4 12B, Qwen3.6-27B, Gemma 4 31B, and Ling-3.0-flash, plus rows for Qwen3.6-27B-MTP, Qwen3.6-27B Q6_K, gpt-oss-120B, Mistral Small 4 119B, and several quant ladders; some runs used vLLM and SGLang.
M1 8GB · llama.cpp · 32,000 ctx
User compares LFM2.5 2.6B and MiniCPM5 2B on small agentic tool-use tasks on an M1 Air. LFM2.5 2.6B runs at 200 t/s prompt and 22 t/s generation with llama.cpp, QAD-Q4_0 GGUF, 32k context, using about 2.5 GB of RAM. MiniCPM5 2B runs at 162 t/s prompt and 16 t/s generation with a DSpark draft model, Q8_0 GGUF, 32k context, q5_1 KV cache, using about 3.8 GB of RAM. User calls LFM2.5 the clear winner: faster, less RAM despite more parameters, and better aligned for English conversation. MiniCPM5 is noted as often responding in Chinese and hallucinating less, but LFM2.5 tends to hallucinate as conversations get longer.
Unknown GPU · llama.cpp
User reports LFM2.5-230M at 192.34 t/s generation and 1611.26 t/s prefill on a Xiaomi 14T Pro phone. Setup is llama.cpp (build 96ffdc41c) running CPU only with 8 threads, the Q4_0 GGUF, and a q8_0 KV cache, measured with llama-bench at 256 prompt and 256 generation tokens over 10 runs.
Unknown GPU · custom engine · 128,000 ctx
User reports a custom inference engine running on a OnePlus 13, pure CPU. The engine is built from scratch, is 450kb, and supports other model architectures.
NVIDIA RTX 4050 6GB · LM Studio · 256,000 ctx
User benchmarks 20 small models on an RTX 4050 6GB. Models tested include Granite 4.1 3B, Gemma-4-agentic-e2b, Nemotron-3-Nano-4B and LFM2.5-8B-A1B. The user selects LFM2.5-1.2B-Instruct as a cheap always-on model.
AMD Strix Halo 128GB · llama.cpp · 128,000 ctx
User reports Liquid AI's LFM2.5-2.6B at 113 tok/s on a Ryzen AI Max+ 395, a CPU speed from the vendor's own benchmarks. Setup is the official Q4_K_M GGUF of around 1.67 GB, a 2.69B parameter model with 128K context, using under 2.5 GB memory in the vendor's tests. The vendor also reports 30 tok/s on a phone and 220 tok/s on an M5 Max, and the user notes independent results are needed. Benchmarks are ToolSandbox 77.83, IFBench 59.17, BFCLv4 56.88 and LiveCodeBench 59.41, and the model card does not recommend it for agentic coding.
On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking