llamaperf

DeepSeek V4.1

on NVIDIA RTX 5090 · streaming-llama-stack

Tone: positiveNVIDIA hardware
Oct 28, 2026
Throughput
7-20 t/s gen
System RAM
32 GB
VRAM reported
32 GB

Use cases

coding

Summary

User reports DeepSeek V4.1 at up to 1770 t/s prefill and 7-20 t/s decode on a single RTX 5090 with 32GB DDR5 and a 25GB/s RAID0 array. Setup is the user's streaming-llama-stack with full-precision weights streamed from NVMe via DirectStorage; decode runs either locally with router biasing toward VRAM-cached experts (20 t/s) or with most experts streamed from disk (7 t/s). Prefill of the whole model takes about 20 seconds at the array's 25GB/s. The stack can also hand KV state over the network to a Mac or large-RAM server for decode. Windows only, Blackwell cards only, and streaming decode currently supports only DeepSeek V4.1.