Qwen3.8 125B (6B active) Flash-Next
on NVIDIA RTX 4060 · llama.cpp
Summary
User reports Qwen3.8-Flash-Next IQ2_XS at 12.5 tok/s on an RTX 4060 8 GB with 24 GB RAM cap, up from 5.2 tok/s stock.
Setup is llama.cpp with moe-cache plugin, IQ2_XS quant, 68 GB model streamed from NVMe with CPU-side expert management.
User also reports Qwen3.6-35B at 22.6 tok/s with 12 GB RAM and GPU (6.9 stock), 9.9 tok/s CPU-only (3.1 stock), GLM-5.3-Flash 102 GB at 1.75 tok/s (0.32 stock), Qwen3.5-122B IQ2_XXS at 7.7 tok/s (1.89 stock), and gpt-oss-120b at 4.47 tok/s (0.37 stock). Notes stock is faster when model fits in RAM (42.9 vs 36.6 tok/s).