Unknown family
on 2× NVIDIA RTX 3070 · Strata · 100,000 ctx
Summary
User reports 40 t/s decode and 320-517 t/s prefill with the current Strata build on two RTX 3070 cards (16GB total VRAM).
Setup uses a custom 100k context setting with MTP depth 3 and 22 CPU threads on a Ryzen 9 3900X with 64GB system RAM.
Prefill is given as a range; 128K context runs out of memory. User says more VRAM and system RAM are needed for larger models and notes two RTX PRO 4500s or an RTX PRO 6000 96GB would cost around 10k-18k.
Compare this run
Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.