DeepSeek V4 Flash 284B (13B active)

on NVIDIA RTX 5090 · llama.cpp · 1,048,576 ctx

Sep 7, 2026

Throughput

21.3 t/s gen · 1105.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.

Quant

Q2_K (GGUF)

System RAM

125 GB

VRAM reported

32 GB

Summary

User reports DeepSeek V4 Flash on an RTX 5090 with a llama.cpp fork, at 22.7 to 21.3 t/s TG and 1105 to 927 t/s PP across prompt lengths. Setup is the Q2_K quant at 1M context, with CPU offloading for the MoE layers.

View source ↗