llamaperf

Edge0 145B

on NVIDIA RTX 3090 · Strata · 16,384 ctx

Tone: mixedNVIDIA hardware
Nov 17, 2026
Throughput
48.6 t/s gen · 1481.2 tokens/s prompt processing (prefill)Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.
Quant
q3k
KV cache
int8
System RAM
116 GB
VRAM reported
24 GB

Use cases

agentictool-use

Summary

User reports Edge0-145B at 48.6 t/s chat decode on a single RTX 3090, with a 16K prompt read at 1,481.2 t/s and agent decode at 52.8 t/s. Setup is Strata with the q3k quant, int8 KV cache, speculative decoding 4, 131K max context, and direct PLE I/O from disk; the FP8 PLE table alone is 51 GB. GPU peak was 23,929 of 24,576 MiB and the engine process held 43.4 GiB of system RAM, so the user does not know the memory floor yet. The agent figure is speed only: three temperature-0 runs gave three different tool calls, one invalid.

Compare this run

Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.

Models most run on the NVIDIA RTX 3090

Bars are medians on one scale, 0 to 111.7 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.

Everything reported on the NVIDIA RTX 3090 →Every Edge0 report →Estimate Edge0 on this card →