Qwen3.8 125B (6B active) Flash-Next
2× NVIDIA Tesla P40 24GB · Strata · 32,768 ctx
- generation:
- 34.8 tokens/s
- prompt processing (prefill):
- 501.0 tokens/s
- quant:
- IQ2_XS (GGUF)
User benchmarks Qwen3.8 Flash-Next at 34.8 tok/s decode on two Tesla P40 24GB cards with a layer split, and 19.6 tok/s on a single P40. Setup is Strata v0.1.39 with IQ2_XS weights at 32,768 context, experimental CUDA 12 engine built locally; the two-card run splits layers across both GPUs (K=24, all 24576 profiled expert pairs resident). Prompt reading was 353 / 501 tok/s at 4K / 16K on both cards and 348 / 388 tok/s on one; figures are medians of 3 runs per cell, with the first run of each configuration slower due to expert-cache warm-up. A 69-task correctness suite ran twice, and two greedy runs of identical prompts gave 49 of 69 identical outputs.