Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.
Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.
Free to use. No account needed. Memory estimates and community measurements are labelled separately.
Search for a model to see its reported speeds across GPUs and Macs.
These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →
Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.
2× NVIDIA RTX 3060 12GB · Strata
User reports Qwen3.8 Flash-Next Coder at 50 t/s generation and 1k t/s prefill on 2x RTX 3060 12GB. Setup is Strata engine with IQ1_M quant, 48 GB system RAM, Ryzen 7 5700X. User built a voxel pagoda as a first test and asks what others are building.
User reports Qwen3.8 Flash-Next at 60 t/s average code generation on empty context, with peaks of 60-70 t/s, on two RTX 3060s. Setup is Strata with IQ3_S weights and MTP speculation enabled; the user tuned --pcie-frac to 0.2, --pool-workers, --trim-stage-weights, --pipeline-windows 6, and forced GPU 1 (the faster PCIe lane) as the main GPU via gpu_order as_given. The user says the previous baseline was about 35 t/s decode peak, and that performance degrades with full context but still holds a solid +15-20% over the old baseline.
User reports Qwen3.8 Flash at 27.9 tok/s on an RTX 3060. Setup is Strata on a single RTX 3060. User finds the model impressive for the GPU but hits a recurring engine error ("layer 1 never rang") after several minutes of use, requiring a reboot, and asks how to troubleshoot it.
On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking