llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX Pro 6000 Blackwell · llama.cpp · 2,048 ctx

NVIDIA hardware
Sep 8, 2026
Throughput
101.9 t/s gen
Quant
IQ4_XS (GGUF)
System RAM
96 GB
VRAM reported
96 GB

Use cases

codinglong-context

Summary

User compares llama.cpp, SGLang, and FreeToken on Qwen3.8-Flash-Next. At full context, TTFT is 35.4s for SGLang, 80.4s for FreeToken, 210.2s for llama.cpp with MTP, and 258.4s for llama.cpp baseline. Decode at full context is 126.9 t/s for SGLang, 87.5 t/s for FreeToken, 52.6 t/s for llama.cpp with MTP, and 20.3 t/s for llama.cpp baseline. MTP improves decode 1.63x at 8K and 1.69x at 32K. Accuracy is 95.22-95.75% on GSM8K and 92.20-93.00% on MATH-500. Startup takes 16s for llama.cpp, 108s for SGLang, and 126s for FreeToken.