Tencent-HY4 Flash
2× NVIDIA RTX 5090 · Basalt
- prompt processing (prefill):
- 3450.0 tokens/s
- quant:
- NVFP4 (NVFP4)
User reports Tencent Hy4 Flash running in Basalt 0.5.0 on an RTX 5090 plus RTX 5060 Ti, at about 84 t/s on prose, 172 t/s on structured output and around 3,450 t/s prefill. Setup is Basalt with an NVFP4 build, a single .basalt file needing roughly 95 to 100 GB of memory total, with more of it in VRAM being better. Speculative decoding uses DFly2. User says throughput is lower than Flash Next because Hy4 Flash has almost twice the weights per token at NVFP4, with more cache misses that are expensive; they expect maybe 10 or 20% more performance to unlock. Quantization hits it hard: the official Q4_K_M measured just below 87% same top 1 against BF16 and the NVFP4 build about 86%, versus above 91% for Flash Next NVFP4. Math felt better on Hy4, and 1M context and dynamic vision both work.