llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX 3090 · llama.cpp

Tone: mixedNVIDIA hardware
Sep 24, 2026
Throughput
40.0 t/s gen · 300.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
IQ4_XS (GGUF)
System RAM
64 GB
VRAM reported
72 GB

Summary

User reports Qwen3.8 Flash-Next at 40 t/s generation and 300 t/s prompt processing on an RTX 3090 in a 4-GPU system totalling 72 GB VRAM. Setup is llama.cpp with an IQ4_XS GGUF. A second configuration with ExLlamaV3 at 3.05 bpw achieves 25 t/s generation and 1,200 t/s prompt processing. The user is considering an Epyc 7443 upgrade with 256 GB DDR4 for more PCIe lanes and CPU offloading, and asks whether the extra RAM and bandwidth would be worthwhile.