llamaperf

Hy4-Flash

5 reports

Run Hy4-Flash yourself? Add your numbers.

Your coding agent can run a speed test on your machine and send it here with a free account. Or paste a result you've already posted.

Reported performance for Hy4-Flash

Filter this model’s reports by setup →
Tone: positiveApple hardware
generation:
36.4 tokens/s
prompt processing (prefill):
968.0 tokens/s
quant:
4-bit (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Hy4-Flash at 36.4 t/s decode and 968 t/s prefill on a MacBook Pro M5 Max with 128 GB, using their own Kapitsa engine. Setup is Kapitsa with a 4-bit MLX quant (8-bit trunk, 4-bit routed experts, group size 64) and the fast profile; the 134 GB pack keeps every expert in RAM with an 88 GB peak, Engram tables memory-mapped on the CPU. Decode falls from 36.4 t/s at short context to 34.6 t/s at 32K; prefill is 657 t/s at 4K on the default profile and 968-971 t/s with the fast profile. MLX 3.1-bit is compared at 20.8 t/s decode and 228 t/s prefill at 4K. A 48 GB M5 Pro streaming experts from SSD decodes at about 3.5 t/s.

Nov 20, 2026
Tone: positiveApple hardware
generation:
36.4 tokens/s
quant:
4-bit (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Hy4-Flash at 36.4 t/s decode on a MacBook Pro M5 Max (40-core GPU, 128 GB), with prefill of 657 tok/s at 4K context in the default profile and 968 tok/s with --profile fast. Setup is the Kapitsa engine with a 4-bit MLX quant (8-bit trunk, 4-bit group-64 routed experts, plus Engram tables), 134 GB on disk and 88 GB peak RAM; the Engram tables are memory-mapped and read on the CPU. Decode falls from 36.4 t/s at short context to 34.6 t/s at 32K, while MLX (mlx-lm 0.32.2, 3.1-bit) drops from 20.8 to 4.4 t/s over the same contexts. Warm turns in a cached conversation start in 0.15-0.21 s at 1K to 32K. A 48 GB M5 Pro streams experts from SSD and decodes at about 3.5 tok/s.

Nov 20, 2026

Hy4-Flash

NVIDIA RTX 3060 12GB · RustingStrata · 20,000 ctx

Tone: mixedNVIDIA hardware
generation:
3.3 tokens/s
prompt processing (prefill):
11.6 tokens/s
quant:
IQ2_S/IQ2_XXS (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

coding

User reports Hy4-flash at 3.3 t/s generation and 11.6 t/s prompt reading on an RTX 3060 12GB with 32 GB of RAM, using a 20,000-token prompt and 64 tokens of output. Setup is RustingStrata with an IQ2_S/IQ2_XXS GGUF of about 85 GB, streamed from an NVMe drive with experts split between CPU and GPU. The Vulkan backend on the same card gave 1.9 t/s generation and 10.6 t/s prompt reading. After a short prompt the model writes 4.3 to 5.1 t/s on CUDA. The user notes one token needs about 2.6 GB of expert weights and calls it not suitable for a coding agent, though the full model runs on a 12 GB card. llama.cpp support was still a draft pull request, so no comparison was made.

Nov 19, 2026

Hy4-Flash

Unknown GPU

Tone: mixed
generation:
10.0 tokens/s
quant:
4-bit

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

long-contextsummarization

User reports Hy4-Flash at 10 t/s on a machine with 26GB VRAM and 128GB DDR4, compared with 15 t/s for QFN. Hy4-Flash ran at 4-bit and QFN at 5-bit; no engine or KV cache setting is stated. The user says Hy4-Flash is substantially more accurate at citing law from PDF documents and calls it the best model for local legal workflows, but describes the speed as poor.

Nov 19, 2026

Hy4-Flash

2× NVIDIA RTX 3090 · llama.cpp · 131,072 ctx

Tone: positiveNVIDIA hardware
generation:
20-24 tokens/s
prompt processing (prefill):
60-65 tokens/s
quant:
Q3_K (GGUF)
kv:
q8_0

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports running Hy4-Flash Q3_K on 2x RTX 3090 with 128 GB DDR4 RAM, getting PP ~60-65 t/s and TG ~20-24 t/s. Setup is llama.cpp (llama-server router mode, a PR build) with Q3_K GGUF weights, q8_0 KV cache and 131072 context, using expert streaming via --moe-cache-mib 33768 and -cmoe, so experts run from system RAM. A logged run shows 4091 prompt tokens at 64.54 t/s and 100 generated tokens at 21.95 t/s. User says the model is slower but finishes answers with fewer tokens, and calls it a likely goto model.

Nov 18, 2026

Get a weekly email of new Hy4-Flash reports on any GPU.

Email me new reports

Reported configurations

Individual observations from this page, not expected speeds or a ranking. Select a model to open its report, with the source and offloading conditions. Reported context may be a limit rather than actual input length.

Model / hardwareQuant / engineReported contextGeneration
Hy4-Flash
M5 Max 128GB
4-bit
Kapitsa
Not reported36.4 tokens/s
Hy4-Flash
M5 Max 128GB
4-bit
Kapitsa
Not reported36.4 tokens/s
Hy4-Flash
NVIDIA RTX 3060 12GB
IQ2_S/IQ2_XXS
RustingStrata
20,0003.3 tokens/s