Unknown family 6.74B
on AMD RX 7900 XTX · llama.cpp
Summary
User benchmarks a llama 7B Q4_0 model (6.74B params, 3.56 GiB) at 172.86 t/s generation and 3290.92 t/s prompt processing on a single AMD RX 7900 XTX.
Setup is llama.cpp (build a81a56957) with the Vulkan backend, flash attention on, ngl 100 and n_batch 512, on Arch Linux with RADV Mesa 25.3.1.
A dual-GPU run with 2x RX 7900 XTX was slower at 129.80 t/s generation and 2546.13 t/s prompt processing, which the user attributes to PCIe transfer overhead on small models.