Llama 2 7B
Intel Arc A770 16GB · llama.cpp
- generation:
- 69.7 tokens/s
- prompt processing (prefill):
- 2074.2 tokens/s
- quant:
- Q4_0 (GGUF)
User benchmarks Llama 2 7B Q4_0 on an Arc A770 16GB with arcint's llama.cpp patches and flash attention on, reaching 2074.21 t/s prefill and 69.71 t/s decode in llama-bench (512 prefill, 128 decode, 5 repetitions). Both builds are at llama.cpp commit 3e4dfa8 on a Ryzen 5700X, Debian 13, xe driver; the A770 sits on PCIe 3.0 x4 behind the chipset with its clock pinned at 2000 MHz. Stock SYCL fp16 with flash attention decodes faster at 73.66 t/s but prefills at 1457.24 t/s; arcint leads prefill by 10.5% on the A770 and 31% on the B60, while SYCL wins decode by 5.4% and 4.4%. Through arcint's own server the A770 gives 1963 / 70.5. On the B60, Qwen3.6-35B-A3B Q4_K_M runs at 2332 / 84.9 and Qwen3.8-27B Q4_K_M at 844 / 19.9; the 27B at full 262,144 context with MTP drafts decodes 47.1 t/s at 3,599 prompt tokens down to 12.5 t/s at 234,553.