llamaperf

Qwen3.8 27B

on NVIDIA RTX 5090 · SGLang

NVIDIA hardware
Oct 8, 2026
Throughput
139.7 t/s gen · 9081.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
NVFP4 (NVFP4)
KV cache
FP8
VRAM reported
32 GB

Summary

User reports Qwen3.8-27B at 139.7 tok/s on an RTX 5090 with SGLang and a D-Flash 2 drafter. Setup is SGLang with NVFP4 weights and an FP8 KV cache, single-user at concurrency 1. Other profiles on the same card: DSpark v2 drafter 110.2 tok/s, native MTP under vLLM 100.8 tok/s, and a vLLM baseline with no speculation at 84.5 tok/s. Prefill is about 9,081 tok/s on the 5090 against about 196 tok/s on an M4 Max 128GB running oMLX 4-bit MLX builds, where decode was 16.7 tok/s with speculation off, 35.5 tok/s with DFlash 2 greedy and 47.2 tok/s with DFlash 2 thinking on. A warm prefix cache cuts time to first token by about 90% on both machines.