NVIDIA RTX A6000 48GB reports

generation:

17.2 tokens/s

quant:

Q8_K_XL

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports DeepSeek-V4-Flash-0731 UD-Q8_K_XL at a steady 17.20 t/s on an RTX A6000 48GB. Setup is an AMD EPYC 74F3 24-core with 8 channel 3200 DDR4. Prompt processing is in the high 70 t/s, down to mid 30 t/s at 300k context. The user says 48GB VRAM is enough for the full 1mil context but prompt processing would be very slow.

generation:

16.9 tokens/s

quant:

bf16 (safetensors)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports bf16 with no quantization at 10.25 GB VRAM and 61 ms TTFT. Source is dev.to Gaurav Vij.