Llama 2 7B
on Intel Arc A770 16GB · llama.cpp
Summary
User benchmarks Llama 2 7B Q4_0 on an Arc A770 16GB with arcint's llama.cpp patches and flash attention on, reaching 2074.21 t/s prefill and 69.71 t/s decode in llama-bench (512 prefill, 128 decode, 5 repetitions).
Both builds are at llama.cpp commit 3e4dfa8 on a Ryzen 5700X, Debian 13, xe driver; the A770 sits on PCIe 3.0 x4 behind the chipset with its clock pinned at 2000 MHz.
Stock SYCL fp16 with flash attention decodes faster at 73.66 t/s but prefills at 1457.24 t/s; arcint leads prefill by 10.5% on the A770 and 31% on the B60, while SYCL wins decode by 5.4% and 4.4%. Through arcint's own server the A770 gives 1963 / 70.5. On the B60, Qwen3.6-35B-A3B Q4_K_M runs at 2332 / 84.9 and Qwen3.8-27B Q4_K_M at 844 / 19.9; the 27B at full 262,144 context with MTP drafts decodes 47.1 t/s at 3,599 prompt tokens down to 12.5 t/s at 234,553.
Compare this run
Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.
Llama 2 7B at 4-bit by hardware
- NVIDIA GTX 1080 Ti11 GB62.0 t/s · one run
Bars are medians on one scale, 0 to 62.0 tokens/s. Press ▶ to watch text arrive at a row's speed.
Models most run on the Intel Arc A770 16GB
- Qwen3 1.7B8-bit33.2 t/s · one run
- 14.4 t/s · one run
- Qwen3.8 27B3-bit12.0 t/s · one run
Bars are medians on one scale, 0 to 33.2 tokens/s. Press ▶ to watch text arrive at a row's speed.
Everything reported on the Intel Arc A770 16GB →Every Llama 2 report →Estimate Llama 2 on this card →