Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.
Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.
Free to use. No account needed. Memory estimates and community measurements are labelled separately.
Search for a model to see its reported speeds across GPUs and Macs.
These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →
Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.
User reports Qwen3.8 Flash-Next at 36.1 t/s on an RTX 4070 Ti Super, up from 30.3 t/s after raising Strata's --adapt-every from 1 to 6. Setup is Strata with UD-Q4_K_XL GGUF on a single card, tested on four Slovak-language chat prompts; the fourth prompt describes 20K tokens of source code. Per-prompt gains ranged from +12% to +55% with UD-Q4_K_XL; a UD-IQ4_XS run reached 63.1 to 71.6 t/s across the same prompts.
NVIDIA RTX 4070 Ti Super · Strata · 131,072 ctx
User reports Qwen3.8 Flash-Next at roughly 45-53 output tokens per second on an RTX 4070 Ti SUPER 16GB. Setup is Strata with Q2 quantization and a 128K context configuration, running on a 16GB GPU with 64GB DDR4 DRAM. The complete workflow used multiple computers; the LLM ran on the 16GB GPU while modelling, image generation and other services were handled elsewhere. The user notes Windows was using 15 to 20 GB of the system RAM.
NVIDIA RTX 4070 Ti Super · Strata · 10,000 ctx
User reports Qwen3.8 Next at 197 t/s generation and 3,300 t/s prefill on an RTX 4070 Ti Super 16GB. Setup is Strata on CachyOS with Q2 weights, 10k and 20k token prompts, 15.6GB VRAM plus 44GB system RAM used. An IQ3XXS run reached 120 t/s generation and 3,000 t/s prefill with 50GB system RAM used; total system power was 350W.
NVIDIA RTX 4070 Ti Super · Strata · 262,144 ctx
User reports Qwen3.8 Flash-Next at 72 t/s generation and ~1,700 t/s prompt reading on an RTX 4070 Ti Super 16GB. Setup is Strata 0.1.39 with IQ3_S (~84GB) at 256K context, on a Ryzen 9 7950X3D with 128GB DDR5. Out of the box it did 8 t/s; after START-HERE.bat --calibrate it reached 72 t/s. A 17K prompt reads in 8 seconds. User's 27-task coding suite scored 27/27 x3 and HumanEval 159/164. Same model family under llama.cpp with offloading ran at 19 t/s. Keeping the 29GB n-gram table in RAM with --ple-io mmap made results worse (25-26/27 and slower).
On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking