Hy4-Flash
on Unknown GPU
Use cases
long-contextsummarization
Summary
User reports Hy4-Flash at 10 t/s on a machine with 26GB VRAM and 128GB DDR4, compared with 15 t/s for QFN.
Hy4-Flash ran at 4-bit and QFN at 5-bit; no engine or KV cache setting is stated.
The user says Hy4-Flash is substantially more accurate at citing law from PDF documents and calls it the best model for local legal workflows, but describes the speed as poor.