llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA V100 32GB · llama.cpp · 262,144 ctx

NVIDIA hardware
Sep 28, 2026
Throughput
27.7 t/s gen · 433.2 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
UD-Q5_K_XL (GGUF)
KV cache
q8_0
VRAM reported
32 GB

Use cases

codingagenticlong-context

Summary

User reports Qwen3.8-27B Flash-Next at 433.19 t/s prompt processing and 27.72 t/s generation on a Tesla V100 32GB, comparing a custom llama.cpp fork against vanilla llama.cpp. Setup is llama.cpp (v100-optimized fork) with UD-Q5_K_XL GGUF, q8_0 K/V cache, 262144 context, FlashAttention on, MTP n-max=3, 63 GPU layers, 100000 cached tokens plus a 1000-token appended prompt. The fork gains +43.94% prompt processing at 100k cached + 1k appended (300.94 to 433.19 t/s) and +32.19% generation with the MTP3 decode stack (27.591 to 36.471 t/s), while standard llama-bench tg128 is effectively unchanged (27.89 to 27.72 t/s).