Edge0 145B
on NVIDIA RTX 3090 · Strata · 16,384 ctx
Use cases
agentictool-use
Summary
User reports Edge0-145B at 48.6 t/s chat decode on a single RTX 3090, with a 16K prompt read at 1,481.2 t/s and agent decode at 52.8 t/s.
Setup is Strata with the q3k quant, int8 KV cache, speculative decoding 4, 131K max context, and direct PLE I/O from disk; the FP8 PLE table alone is 51 GB.
GPU peak was 23,929 of 24,576 MiB and the engine process held 43.4 GiB of system RAM, so the user does not know the memory floor yet. The agent figure is speed only: three temperature-0 runs gave three different tool calls, one invalid.
Compare this run
Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.
Models most run on the NVIDIA RTX 3090
- Qwen3.8 27B4-bit38.4 t/s · 7 runs
- 85.9 t/s · 2 runs
- Gemma 4 31B4-bit36.5 t/s · one run
- Llama 3 8B4-bit111.7 t/s · one run
- Muse 30B4-bit30.0 t/s · one run
- Qwen2.5 32B4-bit28.0 t/s · one run
Bars are medians on one scale, 0 to 111.7 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.
Everything reported on the NVIDIA RTX 3090 →Every Edge0 report →Estimate Edge0 on this card →