Use cases
coding
Summary
User reports DeepSeek V4.1 Flash at up to 103 tok/s writing code on a cluster of two Lucebox workstations, each with a Gorgon Halo 495 128GB plus an R9700 AI PRO 32GB.
The two boxes are joined by a ConnectX-5 Ex card in each with a single 100 GbE cable and no switch, talking over RDMA (RoCE v2) and running as one model, with every token passing through every layer on both boxes. Each step sends about 95 small messages between the boxes.
The user says this is 1.9 times faster than a single Lucebox Zero 495 with 192 GB running the same engine, and that the first two-box run was only a quarter faster than one box. A single 128 GB box cannot keep all of the model's experts in memory, while two boxes have room for it.