Tencent-HY4 Flash
on NVIDIA RTX 3060 12GB · RustingStrata · 20,000 ctx
Summary
User reports Tencent Hy4-flash at 3.3 t/s generation after reading a 20,000-token prompt at 11.6 t/s on an RTX 3060 12GB with 32 GB RAM.
Setup is RustingStrata, the user's own Rust GGUF engine, with an IQ2_S/IQ2_XXS 85 GB GGUF in two files; the model is far larger than the card and the RAM, so weights ran from system memory and NVMe.
The 20,000-token prompt took 29 minutes to read. After a short prompt the model writes 4.3 to 5.1 t/s on CUDA. llama.cpp comparison was not possible because Hy4-flash support there is still a draft pull request.