llamaperf

Hy4-Flash

on NVIDIA RTX 3060 12GB · RustingStrata · 20,000 ctx

Tone: mixedNVIDIA hardware
Nov 19, 2026
Throughput
3.3 t/s gen · 11.6 tokens/s prompt processing (prefill)Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.
Quant
IQ2_S/IQ2_XXS (GGUF)
System RAM
32 GB
VRAM reported
12 GB

Use cases

coding

Summary

User reports Hy4-flash at 3.3 t/s generation and 11.6 t/s prompt reading on an RTX 3060 12GB with 32 GB of RAM, using a 20,000-token prompt and 64 tokens of output. Setup is RustingStrata with an IQ2_S/IQ2_XXS GGUF of about 85 GB, streamed from an NVMe drive with experts split between CPU and GPU. The Vulkan backend on the same card gave 1.9 t/s generation and 10.6 t/s prompt reading. After a short prompt the model writes 4.3 to 5.1 t/s on CUDA. The user notes one token needs about 2.6 GB of expert weights and calls it not suitable for a coding agent, though the full model runs on a 12 GB card. llama.cpp support was still a draft pull request, so no comparison was made.