DeepSeek V2 Lite-Chat
Unknown GPU · Project Zero
- generation:
- 11.7-12.6 tokens/s
- quant:
- Q4_K_S (GGUF)
User reports Project Zero v0.2.0 running DeepSeek-V2-Lite-Chat Q4_K_S at 11.7-12.6 tok/s on a 4-vCPU AVX-512 VNNI Xeon at four threads, up from 6.3-7.3 tok/s. The engine is CPU-only, written in C with no ML framework dependency; the same host recorded llama.cpp at 9.4-10.5 tok/s, and resident memory fell from 12.0 GB to 10.6 GB. With four concurrent clients, aggregate throughput rose from 45 to 101 tok/s for SmolLM2-135M Q8_0 and from 12.7 to 29.8 tok/s for SmolLM2-1.7B Q4_K_M. Startup time dropped to roughly 1-8 seconds.