Minimax M2.1 230B (10B active)
on 2× RTX 4090D 48GB · llama.cpp · 65,536 ctx
Summary
User reports Minimax M2.1 at 20.05 t/s generation and 67.11 t/s prompt eval on an RTX 4090D 48GB plus an RTX 3090.
Setup is llama.cpp build 7542 with UD-Q4_K_XL GGUF at 65536 context, running in router mode on a Xeon W5-3425 with 256GB DDR5.
User notes speed is lower at larger context and says it works great.
Compare this run
Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.
Minimax M2.1 230B · 10B active at 4-bit by hardware
- 25.0 t/s · one run
Bars are medians on one scale, 0 to 25.0 tokens/s. Press ▶ to watch text arrive at a row's speed.