Unknown family
2× NVIDIA RTX 3070 · Strata · 100,000 ctx
- generation:
- 40.0 tokens/s
- prompt processing (prefill):
- 320-517 tokens/s
User reports 40 t/s decode and 320-517 t/s prefill with the current Strata build on two RTX 3070 cards (16GB total VRAM). Setup uses a custom 100k context setting with MTP depth 3 and 22 CPU threads on a Ryzen 9 3900X with 64GB system RAM. Prefill is given as a range; 128K context runs out of memory. User says more VRAM and system RAM are needed for larger models and notes two RTX PRO 4500s or an RTX PRO 6000 96GB would cost around 10k-18k.