Qwen3 30B (3B active)
on NVIDIA RTX 5090 · llama.cpp · 128,000 ctx
Use cases
long-context
Summary
User reports Qwen3-30B-A3B at 79.2 t/s with page-sparse decode at 128k context on an RTX 5090, against 46.1 t/s dense, a 1.72x speedup.
Setup is a llama.cpp fork (sparse-decode branch) with Q4_K_M weights and a q8_0 KV cache, one sequence, 27,648 cached tokens per KV head, five-repeat 64-token llama-bench runs.
Sparse decode reads about 21% of the cache per step and lost 1.0 RULER point versus dense (95% CI -2.2 to -0.02). Prefill is unchanged, VRAM is not saved, and it is CUDA-only and slower below 32k. The 3B Llama-3.2 configuration failed the multi-needle gate at -4.5 points.
Compare this run
Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.
Qwen3 30B · 3B active at 4-bit by hardware
- NVIDIA RTX 509032 GB215.6 t/s · 2 runs
- 148.5 t/s · one run
- 127.7 t/s · one run
- 85.9 t/s · 2 runs
- 73.7 t/s · 3 runs
- 57.0 t/s · one run
Bars are medians on one scale, 0 to 352.1 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.
Models most run on the NVIDIA RTX 5090
- Qwen3.8 27B4-bit80.0 t/s · 7 runs
- 215.6 t/s · 2 runs
- Qwen3.8 27B6-bit69.1 t/s · 2 runs
- 243.9 t/s · one run
- Gemma 4 31B4-bit64.0 t/s · one run
- 563.1 t/s · one run
Bars are medians on one scale, 0 to 563.1 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.
Everything reported on the NVIDIA RTX 5090 →Every Qwen3 report →Estimate Qwen3 on this card →