- generation:
- 2788.0 tokens/s
- quant:
- NVFP4 (NVFP4)
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports 2,788 tokens/s on a single RTX 5090 with LLaDA-2.1-mini + ORL in NVFP4, using the dlmrt inference engine. All numbers are for single-request decoding (batch size 1), not aggregate throughput. The 2,788 t/s figure is a peak result on a specific benchmark, not an average across arbitrary prompts. The post also reports 7,708 tokens/s on 8x B200 with DiffusionGemma 26B MoE + ORL, up to 6.9x higher end-to-end throughput on QwenDLM-4B, and 1,298 tokens/s on a single H100 versus 746 tokens/s for DFlash and 405 tokens/s for EAGLE3.