llamaperf

Qwen3.5 397B (17B active)

on 2× NVIDIA DGX Spark · Ollama

Tone: mixedNVIDIA hardware
Oct 14, 2026
Throughput
15.3 t/s gen
Quant
UD-IQ3_XXS (GGUF)
System RAM
128 GB

Summary

User reports Qwen3.5-397B at 15.3 t/s across two DGX Sparks (128 GB unified each) over llama.cpp RPC, a model that does not fit on one node. Setup is a forked Ollama 0.40.3 with llama.cpp b11351, UD-IQ3_XXS quant (141 GB), 256 output tokens, warm run, two nodes linked by a 200 Gb/s cable. Splitting across nodes costs speed: fast small models lose 30-60% (gemma4:e2b 106.6 to 44.1 t/s), large models lose 15-20%, and load time is 3-5x longer. The default policy keeps a model on one machine and only splits when it does not fit.