llamaperf
Nov 12, 2026
Throughput
30.0 t/s gen
Quant
NVFP4

Use cases

codinglong-context

Summary

User reports about 30 t/s running Qwen3.8 Flash-Next in NVFP4 at 256k context on a laptop with a 16 GB Nvidia GPU and 96 GB DDR5 RAM. Setup details beyond the quant and context length are not stated. The figure is the user's current machine, cited while asking whether to buy a 24 GB VRAM / 128 GB RAM laptop or a 16 GB VRAM / 192 GB RAM laptop for local coding.

Compare this run

Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.

Qwen3.8 125B · 6B active at 4-bit by hardware

Bars are medians on one scale, 0 to 232.0 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.

Models most run on the NVIDIA RTX 5090 Laptop 24GB

Bars are medians on one scale, 0 to 107.0 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.

Everything reported on the NVIDIA RTX 5090 Laptop 24GB →Every Qwen3.8 report →Estimate Qwen3.8 on this card →