Use cases
summarization
Summary
User reports a single B300 GPU running vLLM 0.25.0 at batch 256 with about 300 output tokens per item.
Setup is tensor_parallel_size=1, block_size=256, max_num_seqs=256, enable_prefix_caching=true, moe_backend=flashinfer_trtllm, reasoning_parser=deepseek_v4, attention_config use_fp4_indexer_cache=true, and compilation_config cudagraph_mode=FULL_AND_PIECEWISE.
The user reports deep_gemm_mega_moe requires expert parallel on a single GPU, and that disabling DSpark speculative decoding doubled throughput.