DeepSeek V4.1
on NVIDIA RTX 5090 · streaming-llama-stack
Use cases
codingtool-usevision
Summary
User reports DeepSeek V4.1 at 7.5 t/s decode and over 4000 t/s prefill on an RTX 5090 with 32GB system RAM.
Setup streams full FP4/FP8 weights from a dual PCIe5 512GB RAID0 array, dropping prefill buffers and caching a hot set of experts, with missed experts fetched via DirectStorage.
A slider trades speed for quality: roughly 1.5bpw-equivalent KL divergence gives 24 t/s. The user notes a RAM cache and CPU expert processing would help with more RAM but 32GB is too little.