llamaperf

DeepSeek V2 Lite-Chat

on Unknown GPU · Project Zero

Tone: positive
Oct 24, 2026
Throughput
11.7-12.6 t/s gen
Quant
Q4_K_S (GGUF)

Summary

User reports Project Zero v0.2.0 running DeepSeek-V2-Lite-Chat Q4_K_S at 11.7-12.6 tok/s on a 4-vCPU AVX-512 VNNI Xeon at four threads, up from 6.3-7.3 tok/s. The engine is CPU-only, written in C with no ML framework dependency; the same host recorded llama.cpp at 9.4-10.5 tok/s, and resident memory fell from 12.0 GB to 10.6 GB. With four concurrent clients, aggregate throughput rose from 45 to 101 tok/s for SmolLM2-135M Q8_0 and from 12.7 to 29.8 tok/s for SmolLM2-1.7B Q4_K_M. Startup time dropped to roughly 1-8 seconds.

Compare this run

Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.

Every DeepSeek V2 report →