DeepSeek V4.1
on NVIDIA RTX 5090 · streaming-llama-stack
Use cases
coding
Summary
User reports DeepSeek V4.1 at up to 1770 t/s prefill and 7-20 t/s decode on a single RTX 5090 with 32GB DDR5 and a 25GB/s RAID0 array.
Setup is the user's streaming-llama-stack with full-precision weights streamed from NVMe via DirectStorage; decode runs either locally with router biasing toward VRAM-cached experts (20 t/s) or with most experts streamed from disk (7 t/s).
Prefill of the whole model takes about 20 seconds at the array's 25GB/s. The stack can also hand KV state over the network to a Mac or large-RAM server for decode. Windows only, Blackwell cards only, and streaming decode currently supports only DeepSeek V4.1.