llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX 5070 · llama.cpp

Tone: positiveNVIDIA hardware
Oct 3, 2026
Throughput
11.5 t/s gen
Quant
UD-IQ3_XXS (GGUF)
System RAM
32 GB
VRAM reported
12 GB

Use cases

coding

Summary

User reports Qwen3.8-Flash-Next 177B running at 11-15 tok/s on a single RTX 5070 12GB with 32GB DDR4-2400 RAM and a Ryzen 5 5600GT on Windows. Setup is llama.cpp with UD-IQ3_XXS quant and expert streaming from system RAM, using one file handle per worker and a page-locked hot-expert tier. Benchmark is about 11.5 tok/s, up from roughly 7 tok/s on the inherited setup. Normal conversations run 14-15 tok/s, and a long coding prompt generated 4,892 tokens at 10.15 tok/s. A video shows around 16 minutes for 10k tokens at 10.41 tok/s.