Kimi K3.1 2800B Pro-Coder

User reports Kimi-K3.1-Pro-Coder at 10.8 tok/s single-stream decode and 36.8 tok/s batch/prefill eval on a single RTX PRO 6000 Blackwell (96 GB) with 176 GB RAM in Colab. Setup is the Strata 3-tier MoE offloading engine with dr-triton fused dequant kernels, a 775 GB quant (down from 1.77 TB) with routed experts at 2.81/2.56/1.44 bpw (~1.92 bpw average) and shared experts, attention and embeddings in BF16/F32; experts split across GPU VRAM, pinned RAM and NVMe mmap with router lookahead prefetching at a 97.9% hit rate. The 2.8T model keeps 96.67% on both AIME 2025 and LiveCodeBench (29/30 each), with 95.20% top-1 and 99.92% top-5 token agreement against the base and mean KLD 0.0191. The post also lists projected throughput for other hardware (e.g. ~4-5.5 tok/s decode on a single 4090/5090, ~18-22 tok/s on 2x PRO 6000/H100), which are estimates rather than measurements.