Use cases
rag
Summary
User reports Qwen 3.5 3.5B at 34 t/s generation and 1000+ t/s prompt processing on two RTX 3060 12GB cards.
Setup uses a 4x oculink to mini PCIe 3x connection. User is building a new system with dual 16x lanes and NCCL.
User notes the cards are used for RAG and multiple purposes, and mentions the 12GB 3060 cards are becoming scarce and expensive.