llamaperf

ds4 benchmarks

An inference engine for running open-weight LLMs locally.

15 community reports

ds4 results by GPU

Hardware with reported ds4 results, with each report's model, quant and speed, newest first. Runs on several cards, with speculative decoding, with batched requests or with part of the model in system RAM say so, since each describes a different setup. Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.

This engine doesn't yet have an editorial profile on llamaperf. The community reports below show how it's been used in practice across different hardware.

Compare GPUs running ds4

Every card is compared on one model, Qwen3.8 125B · 6B active at 2-bit, the size and bit class that most cards here have plain runs of. A card with no run of that model shows none rather than a figure from a different model.

GPUVRAMReportsMedian speed, Qwen3.8 125B · 6B active 2-bit
M3 Ultra 512GBapple512GB3no Qwen3.8 125B · 6B active 2-bit run reported
M5 Max 128GBapple128GB3no Qwen3.8 125B · 6B active 2-bit run reported
AMD Strix Halo 128GBamd128GB2no Qwen3.8 125B · 6B active 2-bit run reported
M4 Max 128GBapple128GB2no Qwen3.8 125B · 6B active 2-bit run reported
M5 Pro 64GBapple64GB2no Qwen3.8 125B · 6B active 2-bit run reported
M1 Max 64GBapple64GB1no Qwen3.8 125B · 6B active 2-bit run reported
M3 Max 96GBapple96GB1no Qwen3.8 125B · 6B active 2-bit run reported
M3 Ultra 256GBapple256GB1no Qwen3.8 125B · 6B active 2-bit run reported

ds4 against other engines

Pairs of plain runs on the same card, of the same model size at the same bit class: one device, one request, no speculative decoding, the whole model in memory. Context length and build still differ between the two sides, and each side shows its own.

No matched pair yet. No card has plain runs of one model size at one bit class on ds4 and on another engine, so llamaperf can't say how it compares on speed. Add a run.

Top models on ds4

Frequently asked

Is ds4 faster than other engines?

llamaperf has no matched comparison for ds4 yet: no card has plain runs of the same model size at the same bit class on ds4 and on another engine. Speed claims about engines need that pairing, so this page doesn't make one.