DeepSeek V4 Flash 284B (13B active)

on 4× NVIDIA A40 48GB · TensorSharp · 16,000 ctx

Sep 7, 2026

Throughput

31.5 t/s gen · 836.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.

Quant

Q8_K_XL (GGUF)

Summary

User benchmarks TensorSharp against llama.cpp on 4x A40 GPUs, with TensorSharp's CUDA backend reaching 836 t/s prefill at 16K context and 31.5 t/s decode short. The model is DeepSeek-V4-Flash-0731-UD-Q8_K_XL from unsloth.

View source ↗