DeepSeek V4 Flash 284B (13B active)
on NVIDIA RTX 3090 Ti · llama.cpp · 98,304 ctx
Summary
User reports DeepSeek V4 Flash at about 150 tok/s prompt processing and 15 tok/s generation in short contexts on an RTX 3090 Ti 24GB with 96GB DDR5-6400 and an AMD 9950X CPU.
Setup is llama.cpp with an IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8 GGUF, 98304 context, -ngl 99 and --n-cpu-moe 40, so 40 MoE layers ran on the CPU.
User says it works well in the pi harness with the llama.cpp chat template.