LLaDA-2.1-mini ORL
on NVIDIA RTX 5090 · dlmrt
Summary
User reports 2,788 tokens/s on a single RTX 5090 with LLaDA-2.1-mini + ORL in NVFP4, using the dlmrt inference engine.
All numbers are for single-request decoding (batch size 1), not aggregate throughput. The 2,788 t/s figure is a peak result on a specific benchmark, not an average across arbitrary prompts.
The post also reports 7,708 tokens/s on 8x B200 with DiffusionGemma 26B MoE + ORL, up to 6.9x higher end-to-end throughput on QwenDLM-4B, and 1,298 tokens/s on a single H100 versus 746 tokens/s for DFlash and 405 tokens/s for EAGLE3.