llamaperf

Tencent-HY4 Flash

on 2× NVIDIA RTX 5090 · Basalt

Tone: mixedNVIDIA hardware
Nov 19, 2026
Throughput
3450.0 tokens/s prompt processing (prefill)Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.
Quant
NVFP4 (NVFP4)

Use cases

visionlong-contextmath

Summary

User reports Tencent Hy4 Flash running in Basalt 0.5.0 on an RTX 5090 plus RTX 5060 Ti, at about 84 t/s on prose, 172 t/s on structured output and around 3,450 t/s prefill. Setup is Basalt with an NVFP4 build, a single .basalt file needing roughly 95 to 100 GB of memory total, with more of it in VRAM being better. Speculative decoding uses DFly2. User says throughput is lower than Flash Next because Hy4 Flash has almost twice the weights per token at NVFP4, with more cache misses that are expensive; they expect maybe 10 or 20% more performance to unlock. Quantization hits it hard: the official Q4_K_M measured just below 87% same top 1 against BF16 and the NVFP4 build about 86%, versus above 91% for Flash Next NVFP4. Math felt better on Hy4, and 1M context and dynamic vision both work.