Unknown family 40B
Summary
User reports a 40B model (~25GB) running at 16 t/s across a 3-device cluster: a mini PC with an RTX 3060, a 16GB Mac Mini, and an old laptop.
Setup uses the user's own ramdeck-core pooling repo, sharding the model across the 3060 first, then the Mac's unified memory, then CPU spillover routed to the old laptop.
The full load took about 2.5 minutes and latency was roughly 3.6s. User notes that offloading CPU spillover to the slower laptop beat keeping it on the mini PC's own CPU, and asks whether others have seen the same GPU+CPU contention effect.