llamaperf

Maya-S v2

on 4× NVIDIA V100 32GB · 32,768 ctx

NVIDIA hardware
Oct 21, 2026
Throughput
69.3 t/s gen · 622.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
IQ2_XXS (GGUF)
System RAM
126 GB
VRAM reported
128 GB

Summary

User reports Maya-S-v2-IQ2_XXS at 69.3 tok/s decode on four Tesla V100-PCIE-32GB cards, with prefill of 622 tok/s at 8192 tokens and 466 tok/s at 2048. Setup is engine v1.1.19 at 32K context, with MTP speculative decoding on (589 of 750 drafts accepted, 78.5%) and the whole model resident in VRAM across 128 GB. The 69.3 tok/s is the mean of 3 greedy runs with thinking off; the engine's own stat logged 14.52 ms/token (68.86 tok/s) at a 100% VRAM hit rate. The split leaves about 33 GB of VRAM unused, and the 96.5 GB model loads from a SATA SSD in about 180 s. These are the engine's own split numbers, before --calibrate.