Qwen2.5 0.5B
on Unknown GPU · llama.cpp · 512 ctx
Summary
User reports Qwen2.5-0.5B-Instruct at 0.63 tok/s on a 2006 Dell Latitude D620 laptop with no GPU.
Setup is llama.cpp on CPU with Q4_0 GGUF, 512 context, 2 threads, 128 tokens generated; the machine has an Intel Core Duo T2300 and 2 GB DDR2 RAM.
A single run took 203.2 s with 41.2 s to first token, and 1024 context crashed with OOM. The user does not recommend it and calls the results not generalizable.