Summary
User reports Qwen3.8-Flash-Next 177B NVFP4 at 9.06 tok/s decode on a single RTX 5060 Ti 16GB with 32GB DDR5, using a custom SSD-streaming engine that keeps 99 GiB of the 119 GiB model on NVMe.
Setup streams routed experts and a 50.7 GiB n-gram table from a Gen5 NVMe SSD on 8 read threads, with about 270 MiB read per token and roughly 75% of expert lookups hitting VRAM or RAM. Prefill is 49.2 tok/s on a 5.5k-token prompt.
Best decode turn reached 10.4 tok/s; llama.cpp on the same machine averaged 4.9 tok/s. The same engine runs Qwen3.6-35B-A3B NVFP4 at 47.3 tok/s decode and 591 tok/s prefill. v1 is limited to RTX 50-series on Windows 11 and WSL2 with greedy decoding only.