llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX 5070 Ti · EuLLM

Tone: mixedNVIDIA hardware
Oct 6, 2026
Throughput
54.9 t/s gen
Quant
IQ2_XS (GGUF)
System RAM
64 GB
VRAM reported
16 GB

Summary

User reports Qwen3.8-Flash-Next at 54.9 tok/s decode on an RTX 5070 Ti 16 GB, with 33 GiB of experts kept in RAM. Setup is EuLLM (llama.cpp b11370 + PR #29887) with IQ2_XS, 2048-token micro-batch, and a GPU expert cache; the model is 125B total with about 6B active per token. The same PC reaches 85.8 tok/s with Strata and MTP, and prompt processing peaks at 1,500–1,743 tok/s with prefetch versus 2,320 tok/s for Strata. MTP loses 27% on this model with 52% draft acceptance, and gains only +7% prose / +16% code on Qwen3.6-35B-A3B.