llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX 3060 12GB · llama.cpp

Tone: positiveNVIDIA hardware
Oct 11, 2026
Throughput
20.1-21.1 t/s gen
Quant
IQ2_XS (GGUF)
System RAM
16 GB
VRAM reported
12 GB

Summary

User reports Qwen3.8 Flash-Next 125B-A6B at 20.14-21.13 tok/s on an RTX 3060 12GB with 16GB DDR4 RAM, using a custom llama.cpp build with --moe-direct-io prefetching. Setup is llama.cpp with IQ2_XS quant and a 68GB model file, run under a 6G cgroup memory limit; the model is streamed from NVMe SSD with no expert pruning and bit-exact output. Stock llama.cpp on the same machine gives 1.41-2.12 tok/s with 1,140-1,565 major page faults per token; warm cache runs reach 24+ tok/s, cold starts begin at 3.5-4.5 tok/s, and prompt processing remains slow.