Hy4-Flash
on 2× NVIDIA RTX 3090 · llama.cpp · 131,072 ctx
Summary
User reports running Hy4-Flash Q3_K on 2x RTX 3090 with 128 GB DDR4 RAM, getting PP ~60-65 t/s and TG ~20-24 t/s.
Setup is llama.cpp (llama-server router mode, a PR build) with Q3_K GGUF weights, q8_0 KV cache and 131072 context, using expert streaming via --moe-cache-mib 33768 and -cmoe, so experts run from system RAM.
A logged run shows 4091 prompt tokens at 64.54 t/s and 100 generated tokens at 21.95 t/s. User says the model is slower but finishes answers with fewer tokens, and calls it a likely goto model.