Qwen3.8 125B (6B active) Flash-Next
on 2× NVIDIA V100 16GB · llama.cpp
Oct 16, 2026
Use cases
coding
Summary
User asks whether NInfer can run on a V100 16GB plus RTX 4070 12GB, reporting 50 t/s on llama.cpp and 35 t/s on coding at long context.
Setup is llama.cpp with an IQ3_XXS Flash-Next quant; the machine has 32GB of system RAM.
The user also gets a sustained 40 t/s on Strata with the same IQ3_XXS Flash-Next, which they find performs better than a Q4 27B, but the 32GB of RAM causes slow prefill and occasional freezing.