Qwen3.5 397B (17B active)
on 2× NVIDIA DGX Spark · Ollama
Summary
User reports Qwen3.5-397B at 15.3 t/s across two DGX Sparks (128 GB unified each) over llama.cpp RPC, a model that does not fit on one node.
Setup is a forked Ollama 0.40.3 with llama.cpp b11351, UD-IQ3_XXS quant (141 GB), 256 output tokens, warm run, two nodes linked by a 200 Gb/s cable.
Splitting across nodes costs speed: fast small models lose 30-60% (gemma4:e2b 106.6 to 44.1 t/s), large models lose 15-20%, and load time is 3-5x longer. The default policy keeps a model on one machine and only splits when it does not fit.