Qwen3.8 27B
2× NVIDIA RTX 5060 Ti 16GB · SGLang
- generation:
- 126.0 tokens/s
- quant:
- NVFP4 (NVFP4)
- kv:
- FP8
User reports Qwen3.8-27B at approximately 126 tok/s single-stream decode on two RTX 5060 Ti 16 GB cards. Setup is SGLang 0.5.23 with NVFP4 weights, FP8 KV cache, tensor parallelism TP=2, and a DFlash2 drafter with 8 draft tokens; the KV pool holds 126,585 tokens against a 131,072 advertised context. Decode falls to 107.4 tok/s at 60K context (2,681 tok/s prefill) and 87.3 tok/s at 124K (1,577 tok/s prefill); a rerun at 126,000 context gave 124 and 85 tok/s. Across about a dozen DFlash configurations the short-prompt median stayed between 120 and 128 tok/s, and the two cards draw roughly 220-270 W during decode.