SGLang on the NVIDIA RTX 3090
llamaperf has 5 SGLang reports on the NVIDIA RTX 3090, none of them a plain single-card run with a model size and quant, so there is no median to quote. The reports are listed below with how each one ran.
5 SGLang reports on the NVIDIA RTX 3090
Newest first. A run on several cards, with speculative decoding, with part of the model in system RAM or with batched requests says so, since each describes a different setup.
- Qwen3.8 27B · W8A8PP 3843.0 t/s(4 cards, speculative decoding)
- Qwen3.8 27B · 3.00bpwPP 904.0 t/s(speculative decoding, part in system RAM)
- Qwen3.8 27B · EXL3 3.0 bpw96.2 t/s(speculative decoding)
- Qwen3.8 27B · AWQ-INT4136.8 t/s(4 cards, speculative decoding)
- Qwen3.8 27B · 3.00bpw98.2 t/s(speculative decoding)
Frequently asked
How fast is SGLang on the NVIDIA RTX 3090?
llamaperf has 5 SGLang reports on the NVIDIA RTX 3090, none of them a plain single-card run with a model size and quant, so there is no median to quote. The reports are listed below with how each one ran.