llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX 4070 · Nebula · 24,576 ctx

NVIDIA hardware
Sep 12, 2026
Throughput
7.2 t/s gen
System RAM
128 GB
VRAM reported
12 GB

Summary

User reports Qwen3.8-Flash-Next at 7.19 t/s on an RTX 4070 Ti 12GB with 128GB DDR4 RAM. Setup is the Nebula C/CUDA engine with native MTP speculative decoding, GPU expert caching (27 of 512 experts per layer resident in VRAM), and CPU MoE execution on an Intel i9-9940X using 14 threads, with a 24,576-token context capacity. A limited-tolerance acceptance mode reaches 7.54 t/s. Time to first token is 25.11 seconds for a 512-token input and 113.94 seconds for 2,048 tokens. Prefill is noted as slow for longer prompts.