NVIDIA RTX A6000 48GB

NVIDIA · 48GB · 2 reports

On the NVIDIA RTX A6000 48GB, Gemma 4 5.1B at 16-bit, the model most run on it, generates a median of 16.9 tokens per second from one plain run (community reports, as of 30 Oct 2026).

As of 30 Oct 2026, the models most run on the NVIDIA RTX A6000 48GB, with generation and prompt-processing medians from plain runs (one device, one request, no speculative decoding):

Gemma 4 5.1B
16.9gen t/s at 16-bit · one run

Engines people use on it: llama.cpp 1 (reports)

See reported performance Estimate what fits

How to compare these reports →Filter reports by setup →Best local LLMs for 48 GB of VRAM →

Run models on your NVIDIA RTX A6000 48GB? Add your numbers.

Your coding agent can run a speed test on your machine and send it here with a free account. Or paste a result you've already posted.

Benchmark with your agent Paste a result

Reported performance on the NVIDIA RTX A6000 48GB

These reports mix GPU counts and memory setups, so a model listed here may have needed extra cards or CPU offloading. The calculator estimates which models fit in 48 GB of VRAM.

This page is thin (2 of 3 reports needed for indexing). Help fill it in.

Date ↓Date ↑Token speed ↓Token speed ↑Prefill ↓Prefill ↑Context ↓Context ↑Ranked

DeepSeek V4 Flash 284B (13B active)

NVIDIA RTX A6000 48GB · 1,000,000 ctx

generation:: 17.2 tokens/s
quant:: Q8_K_XL

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports DeepSeek-V4-Flash-0731 UD-Q8_K_XL at a steady 17.20 t/s on an RTX A6000 48GB. Setup is an AMD EPYC 74F3 24-core with 8 channel 3200 DDR4. Prompt processing is in the high 70 t/s, down to mid 30 t/s at 300k context. The user says 48GB VRAM is enough for the full 1mil context but prompt processing would be very slow.

Gemma 4 5.1B E2B

NVIDIA RTX A6000 48GB · llama.cpp

generation:: 16.9 tokens/s
quant:: bf16 (safetensors)

text-generation

User reports bf16 with no quantization at 10.25 GB VRAM and 61 ms TTFT. Source is dev.to Gaurav Vij.

Get a weekly email of new NVIDIA RTX A6000 48GB reports and newly released models that fit it.

Email me new reports

Reported configurations

Individual observations from this page, not expected speeds or a ranking. Select a model to open its report, with the source and offloading conditions. Reported context may be a limit rather than actual input length.

Model / hardware	Quant / engine	Reported context	Generation
DeepSeek V4 Flash 284B (13B active) NVIDIA RTX A6000 48GB	Q8_K_XL Engine not reported	1,000,000	17.2 tokens/s
Gemma 4 5.1B E2B NVIDIA RTX A6000 48GB	bf16 llama.cpp	Not reported	16.9 tokens/s

Frequently asked

How fast is the NVIDIA RTX A6000 48GB for local LLMs?

On the NVIDIA RTX A6000 48GB, Gemma 4 5.1B at 16-bit, the model most run on it, generates a median of 16.9 tokens per second from one plain run (community reports, as of 30 Oct 2026).