llamaperf

Tencent-HY4 Flash

on NVIDIA RTX 3060 12GB · RustingStrata · 20,000 ctx

NVIDIA hardware
Nov 19, 2026
Throughput
3.3 t/s gen · 11.6 tokens/s prompt processing (prefill)Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.
Quant
IQ2_S/IQ2_XXS (GGUF)
System RAM
32 GB
VRAM reported
12 GB

Summary

User reports Tencent Hy4-flash at 3.3 t/s generation after reading a 20,000-token prompt at 11.6 t/s on an RTX 3060 12GB with 32 GB RAM. Setup is RustingStrata, the user's own Rust GGUF engine, with an IQ2_S/IQ2_XXS 85 GB GGUF in two files; the model is far larger than the card and the RAM, so weights ran from system memory and NVMe. The 20,000-token prompt took 29 minutes to read. After a short prompt the model writes 4.3 to 5.1 t/s on CUDA. llama.cpp comparison was not possible because Hy4-flash support there is still a draft pull request.