llamaperf
Oct 22, 2026
Throughput
7.4 t/s gen
Quant
2-bit
System RAM
64 GB
VRAM reported
12 GB

Use cases

codingagentic

Summary

User reports DeepSeek-V4.1-Flash-1017 at 7.4 tok/s with a warm cache and 5.8 tok/s cold on an RTX 3060 12GB with 64 GB DDR4 and NVMe storage. Setup is a custom engine that computes tokens on a 2-bit copy of the experts, recomputes flagged tokens with the original FP4 weights, and uses a draft head proposing blocks of five tokens. The 510 GB model streams from disk; plain exact decoding on the same machine gives 2.14 tok/s. Reasoning and agent prompts run at 4.4 to 5.2 tok/s. On 960 audited tokens the 2-bit copy matches the exact path 97.4% of the time, 98.3% with corrections; a 1,101-token agent prompt scores 95.9%. The engine is not yet released.