llamaperf

Unknown family

on 2× NVIDIA RTX 4090 · Strata · 524,288 ctx

NVIDIA hardware
Nov 14, 2026
Throughput
25.0 t/s gen
Quant
Q4_K_XL (GGUF)

Summary

User asks whether moving a helper RTX 2080 Ti from PCIe 3.0 x8 to a spare PCIe 3.0 x4 slot would improve token generation, with the RTX 4090 moving to full PCIe 4.0 x16. Current setup is Strata with a Q4_K_XL quant at 512k context and default parameters, on a 5800X3D AM4 board. User reports roughly 100 t/s prefill and 25 t/s generation, and asks whether the slot change would help or whether full tuning is needed. The model is not named.

Compare this run

Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.

Models most run on the NVIDIA RTX 4090

Bars are medians on one scale, 0 to 158.9 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.

Everything reported on the NVIDIA RTX 4090 →