Strata on the NVIDIA RTX 5090
llamaperf has 4 Strata reports on the NVIDIA RTX 5090, none of them a plain single-card run with a model size and quant, so there is no median to quote. The reports are listed below with how each one ran.
4 Strata reports on the NVIDIA RTX 5090
Newest first. A run on several cards, with speculative decoding, with part of the model in system RAM or with batched requests says so, since each describes a different setup.
- Qwen3.8 125B · 6B active · NVFP4 GPTQ162.9 t/s(speculative decoding, part in system RAM)
- Qwen3.8 125B · 6B active · IQ3_S161.3 t/s(speculative decoding, part in system RAM)
- Qwen3.8 125B · 6B active · IQ3_XXS193.8 t/s(speculative decoding, part in system RAM)
- Qwen3.8 125B · 6B active · IQ3_XXSPP 1796.0 t/s(2 cards, speculative decoding, part in system RAM)
Frequently asked
How fast is Strata on the NVIDIA RTX 5090?
llamaperf has 4 Strata reports on the NVIDIA RTX 5090, none of them a plain single-card run with a model size and quant, so there is no median to quote. The reports are listed below with how each one ran.