llamaperf

DeepSeek V4 Flash 284B (13B active)

on NVIDIA RTX Pro 6000 Blackwell · ds4 · 2,048 ctx

Tone: mixedNVIDIA hardware
Oct 9, 2026
Throughput
45.9 t/s gen · 362.3 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
q2-imatrix (GGUF)
System RAM
50 GB
VRAM reported
96 GB

Use cases

agentictool-uselong-context

Summary

User reports DeepSeek-V4-Flash q2-imatrix at 45.9 t/s generation and 362.3 t/s prefill on an RTX Pro 6000 Blackwell 96GB at 2048 context. Setup is ds4 with q2-imatrix GGUF, fp16 dequant cache disabled and MTP off, fully resident at ~91 GB peak. The 80.76 GB model only fits with DS4_CUDA_Q8_F16_CACHE_MB=0; MTP adds 3.5 GB and pushes over 96 GB. Generation beats posted M5 Max and DGX Spark GB10 q2 numbers; prefill is about equal to Spark. At 28672 context generation drops to 40.6 t/s. The user notes the card is tight for this model and heavier ds4_test sessions OOM.