- generation:
- 28.9 tokens/s
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports a 7B Qwen2.5 model at 28.87 tok/s on an RTX 5050 Laptop 8GB in an MSI laptop with an Intel i7 240H and 32 GB DRAM, with 0.53 s time to first token. With RAG retrieval in the loop, generation drops to 25.18 tok/s and TTFT rises to 2.35 s including the knowledge search. A legacy LG Gram with an Intel i7-6500U, 8 GB DRAM and integrated graphics is used for comparison, running 0.5B to 1.5B models. Quantisation format and context length are not stated.