ExLlamaV3 on the NVIDIA RTX 3090
llamaperf has 6 ExLlamaV3 reports on the NVIDIA RTX 3090, none of them a plain single-card run with a model size and quant, so there is no median to quote. The reports are listed below with how each one ran.
6 ExLlamaV3 reports on the NVIDIA RTX 3090
Newest first. A run on several cards, with speculative decoding, with part of the model in system RAM or with batched requests says so, since each describes a different setup.
- GLM-5.3 320B · 18B active · exl3-3.05bpw28.6 t/s(part in system RAM)
- GLM-5.3 320B · 18B active · 3.05bpw17.3 t/s(part in system RAM)
- Qwen3.8 27B · exl3-5.00bpw60.0 t/s
- Qwen3.8 27B · EXL3-4.00bpw162.9 t/s(speculative decoding)
- Qwen3.8 125B · 6B active · 3bpw80.0 t/s(3 cards, speculative decoding)
- Qwen3.8 125B · 6B active · 3.05bpw110.0 t/s(3 cards)
Frequently asked
How fast is ExLlamaV3 on the NVIDIA RTX 3090?
llamaperf has 6 ExLlamaV3 reports on the NVIDIA RTX 3090, none of them a plain single-card run with a model size and quant, so there is no median to quote. The reports are listed below with how each one ran.