Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.
Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.
Free to use. No account needed. Memory estimates and community measurements are labelled separately.
Search for a model to see its reported speeds across GPUs and Macs.
These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →
Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.
4× NVIDIA V100 16GB · NInfer · 256,000 ctx
User reports Qwen3.8 27B at 353 t/s on four V100 16GB cards. Setup is NInfer with NVFP4 weights, Q8 KV cache, 256k max context and MTP 4/0 speculation, tensor parallel across 4 GPUs. The user previously ran 2 V100s at 181 t/s and notes the repo now supports tp4.
2× NVIDIA V100 16GB · NInfer · 180,000 ctx
User reports Qwen3.8-27B NVFP4 at 143.5 committed output tok/s on two Tesla V100-SXM2 16GB cards with an 85,000-token coding input. Setup is NInfer-V100X2 with NVFP4 weights, INT8 group-64 KV cache, MTP4, 180,000-token context capacity, PCIe P2P and no NVLink. Prefill averaged about 1,310 tok/s over the 85K input, with peaks around 2,500 tok/s possible under PP2 pipeline parallelism. Short structured JSONL generation with DFlash7 reached 253+ tok/s, averaging 255.3 tok/s over the final two runs with 118 input tokens and 99.05% draft acceptance. The 143 tok/s route uses an experimental attention implementation that can change outputs and is off by default; on 104 GPQA-Diamond questions with 85K padding it scored 90/104 versus 88/104 for default attention.
2× NVIDIA V100 16GB · NInfer
User reports Qwen3.8 27B at 180 t/s on a pair of V100 16GB cards without NVLink. Setup uses NInfer with NVFP4 weights, Q8 KV cache and MTP 4/0 speculative decoding. The user calls it the legend of V100 and links the tuxKOH/ninfer-V100X2 repo.
On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking