NVIDIA RTX A4000 16GB

NVIDIA · 16GB · 2 reports

Engines people use on it: llama.cpp 1 (reports)

See reported performance Estimate what fits

How to compare these reports →Filter reports by setup →Best local LLMs for 16 GB of VRAM →

Run models on your NVIDIA RTX A4000 16GB? Add your numbers.

Your coding agent can run a speed test on your machine and send it here with a free account. Or paste a result you've already posted.

Benchmark with your agent Paste a result

Reported performance on the NVIDIA RTX A4000 16GB

These reports mix GPU counts and memory setups, so a model listed here may have needed extra cards or CPU offloading. The calculator estimates which models fit in 16 GB of VRAM.

This page is thin (2 of 3 reports needed for indexing). Help fill it in.

Date ↓Date ↑Token speed ↓Token speed ↑Prefill ↓Prefill ↑Context ↓Context ↑Ranked

Qwen3.8 27B TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NEO-CODER-MAX

NVIDIA RTX A4000 16GB

generation:: 30.0 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.8 27B at about 30 t/s on an RTX A4000 VM. The model is the DavidAU TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NEO-CODER-MAX-MTP GGUF. The VM costs $0.15/hour and is shared with a friend; the user offers to move to an RTX A6000 if more people join.

Qwen3.8 27B

3× NVIDIA RTX A4000 16GB · llama.cpp · 128,000 ctx

generation:: 23.0 tokens/s
quant:: Q8_0 (GGUF)

coding

User reports Qwen3.8 27B at ~23 t/s on 3x RTX A4000 16GB. Setup is llama.cpp with Q8_0 quantization, MTP draft mode and ngram speculative decoding. User asks for optimization advice.

Get a weekly email of new NVIDIA RTX A4000 16GB reports and newly released models that fit it.

Email me new reports

Reported configurations

Individual observations from this page, not expected speeds or a ranking. Select a model to open its report, with the source and offloading conditions. Reported context may be a limit rather than actual input length.

Model / hardware	Quant / engine	Reported context	Generation
Qwen3.8 27B TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NEO-CODER-MAX NVIDIA RTX A4000 16GB	Not reported Engine not reported	Not reported	30.0 tokens/s
Qwen3.8 27B 3× NVIDIA RTX A4000 16GB	Q8_0 llama.cpp	128,000	23.0 tokens/s