llamaperf
Sep 7, 2026
Throughput
7-8 t/s gen

Summary

User reports 7-8 t/s with 2x3060 and RAM offloading, but 4.5-5.5 t/s with 3x3060. User asks why adding a GPU slows down generation, and notes there is no tensor parallelism in both chats. User asks about enabling ngram or mlock.