llamaperf

Qwen3.8 125B (6B active) Flash-Next

on 2× NVIDIA RTX 5070 Ti · capy-flash

Tone: positiveNVIDIA hardware
Oct 16, 2026
Throughput
100.0 t/s gen · 3400.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
NVFP4 (NVFP4)
System RAM
96 GB

Summary

User reports Qwen3.8-Flash-Next 125B-A6B at ~100 tok/s generation and ~3.4k tok/s prefill on a 15k prompt on 2x RTX 5070 Ti 16GB. Setup is the capy-flash engine with NVFP4 weights, experts offloaded to RAM, tensor parallel across both GPUs, on a Ryzen 9 9900X with 96 GB DDR5. User compares against Strata, which gets 78 tok/s generation and 2.7k tok/s prefill with the same NVFP4 weights; Strata's 3-bit IQ3_S build is still faster at generation. User also notes a tensor-parallel finding: writing partial sums in FP16 instead of BF16 restores single-GPU numerical parity at no real speed cost.