llamaperf

Qwen3.5 122B (10B active)

on 2× NVIDIA RTX PRO 5000 Blackwell (one 72GB card and one 48GB card) · llama.cpp

Oct 13, 2026
Throughput
112.9 t/s gen · 1234.5 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
Q6_K_L (GGUF)
KV cache
F16
Flash Attention
on
System RAM
96 GB

Summary

User reports Qwen3.5 122B-A10B at 112.9 t/s generation and 1234.5 t/s prompt processing on one RTX PRO 5000 Blackwell 72 GB plus one RTX PRO 5000 Blackwell 48 GB. Setup is llama.cpp CUDA b10941 with Q6_K_L, f16 K/V cache, flash attention on, -ngl 999, batch 2048 and microbatch 512, split tensor 3/2 across the two cards. The 112.9 t/s is TG128 and the 1234.5 t/s is PP512, separate tests rather than one combined request, one repetition per cell; the same run's PP2048 and PP8192 reached 2092.1 t/s and 2262.2 t/s.