llamaperf

Unknown family

on 2× NVIDIA RTX 3070 · Strata · 100,000 ctx

Tone: mixedNVIDIA hardware
Oct 27, 2026
Throughput
40.0 t/s gen · 320-517 tokens/s prompt processing (prefill)Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.
System RAM
64 GB
VRAM reported
16 GB

Summary

User reports 40 t/s decode and 320-517 t/s prefill with the current Strata build on two RTX 3070 cards (16GB total VRAM). Setup uses a custom 100k context setting with MTP depth 3 and 22 CPU threads on a Ryzen 9 3900X with 64GB system RAM. Prefill is given as a range; 128K context runs out of memory. User says more VRAM and system RAM are needed for larger models and notes two RTX PRO 4500s or an RTX PRO 6000 96GB would cost around 10k-18k.

Compare this run

Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.

Everything reported on the NVIDIA RTX 3070 →