Unknown family 27B
on NVIDIA RTX 3060 12GB · RAMDeck
Summary
User reports a 27B model at about 16 GB sharded across 4 devices via RAMDeck, reaching 1.92 tok/s at roughly 25 ms latency.
The devices are an old 12 GB Windows laptop as primary with 3.4 GB, a Mini PC with an RTX 3060 with 20 GB, a Mac mini with 3.7 GB, and an Android phone with 1 GB. The model family is not named.
The user notes the run is slower than a prior 13B run and emphasizes feasibility over speed.