GLM-4.5-Air 106B (12B active)
on 4× NVIDIA RTX Pro 6000 Blackwell · SGLang · 128,000 ctx
Oct 8, 2026
Summary
User reports GLM-4.5-Air FP8 at 133 tokens/sec with SGLang on an RTX 6000 Pro Blackwell, compared with about 78 tokens/sec in vLLM.
Setup is SGLang with FP8 weights, fp8_e5m2 KV cache, 128000 context length and tensor parallelism across 4 GPUs.
The user asks how to reach the same 133 tokens/sec in vLLM with Triton, and whether the Cutlass design is slower than Triton.