llamaperf

Qwen3.8 125B (6B active) Flash-Next

on 2× RTX 4090 D 48GB · Strata · 262,144 ctx

Oct 3, 2026
Throughput
125.2 t/s gen · 4112.6 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
NVFP4 (NVFP4)
KV cache
INT8
System RAM
128 GB

Use cases

codinglong-context

Summary

User reports Qwen3.8-Flash-Next NVFP4 at 125.17 tok/s decode and 4,112.65 tok/s prefill on a dual-GPU setup with an RTX 4090 D 48GB and an RTX 5070 Ti 16GB. Setup is the Strata engine 0.1.35 with an experimental dual-GPU NVFP4 fork, INT8 KV cache, 262,144-token context, and MTP K4 speculative decoding. The 4090 D handled dense layers, KV cache, prefill, output head and MTP, while the 5070 Ti stored 4,500 additional routed experts. The figures come from a single-run capacity test with a synthetic 256,018-token prompt built by repeating code-review material, which the user notes had unusually favorable locality and should be treated as a best-case result. An abliterated NVFP4 checkpoint reached 92.05 tok/s decode and 4,071.39 tok/s prefill with 69.7% draft acceptance. During real coding-agent workloads the NVIDIA model produced around 108-144 tok/s.