Maya-S v2
on 4× NVIDIA V100 32GB · 32,768 ctx
Summary
User reports Maya-S-v2-IQ2_XXS at 69.3 tok/s decode on four Tesla V100-PCIE-32GB cards, with prefill of 622 tok/s at 8192 tokens and 466 tok/s at 2048.
Setup is engine v1.1.19 at 32K context, with MTP speculative decoding on (589 of 750 drafts accepted, 78.5%) and the whole model resident in VRAM across 128 GB.
The 69.3 tok/s is the mean of 3 greedy runs with thinking off; the engine's own stat logged 14.52 ms/token (68.86 tok/s) at a 100% VRAM hit rate. The split leaves about 33 GB of VRAM unused, and the 96.5 GB model loads from a SATA SSD in about 180 s. These are the engine's own split numbers, before --calibrate.