DeepSeek V4.1

generation:

7-20 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports DeepSeek V4.1 at up to 1770 t/s prefill and 7-20 t/s decode on a single RTX 5090 with 32GB DDR5 and a 25GB/s RAID0 array. Setup is the user's streaming-llama-stack with full-precision weights streamed from NVMe via DirectStorage; decode runs either locally with router biasing toward VRAM-cached experts (20 t/s) or with most experts streamed from disk (7 t/s). Prefill of the whole model takes about 20 seconds at the array's 25GB/s. The stack can also hand KV state over the network to a Mac or large-RAM server for decode. Windows only, Blackwell cards only, and streaming decode currently supports only DeepSeek V4.1.

generation:

9.0 tokens/s

prompt processing (prefill):

150.0 tokens/s

quant:

IQ2

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports DeepSeek V4.1 at 9 t/s decode average on an RX 7900 XTX 24GB plus W6800 32GB, with prefill around 150 t/s. Setup is Dspark with mostly IQ2 weights and Q6 n-grams, about 350GB on disk, streamed from a PCIe gen3 NVMe drive on a Ryzen 9 3950X with 96GB dual-channel DDR4 at 3000MHz. Decode varied widely: peaks of 19 t/s, usually around 13 t/s, dropping to 3 t/s when I/O was needed. User calls the model very smart but says it thinks a lot, so the slow average limits it to chatting.