Local model performance on your hardware

generation:

249.1 tokens/s

prompt processing (prefill):

17982.0 tokens/s

kv:

fp8

User reports DeepSeek-V4.1-Flash at 249.1 tok/s single-stream decode and 17,982 tok/s prefill on four RTX PRO 6000 Max-Q cards. Setup is vLLM with bfloat16 weights, fp8 KV cache, TP4/DCP1, RAM Engram tables, adaptive DSpark K7 speculation, 4096-token budget, 32 slots and a 131,072 context cap. C8 aggregate output is 805.5 tok/s and the C1 verifier rate is 98.58 steps/s; the previous Karmic Kraken release measured 230.6 tok/s, 748.3 tok/s, 17,503 tok/s and 89.87 steps/s respectively.

generation:

177.0 tokens/s

quant:

NVFP4

User compares Flash-Next against a dense 27B model. Flash-Next is faster and mechanically flawless, with zero failures on strict JSON, injection resistance and SLAs, and wins the high-reasoning spatial and code-gen tier with better failure modes. The dense 27B still wins sustained multi-step symbolic work such as bug-fixing, math proofs and abstract puzzles. On that symbolic work Flash-Next shows a new failure shape: it promises the deliverable, declares "done", and outputs nothing. Both models use the same reasoning_effort knob with radically different semantics, so Flash-Next is not a drop-in replacement but a conditional promotion.