llamaperf

Bonsai 2 27B

on Intel Arc B580 12GB · llama.cpp · 131,072 ctx

Tone: positiveIntel hardware
Oct 8, 2026
Throughput
42.3 t/s gen · 895.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
PTQ1_0 (GGUF)
KV cache
q4_0
VRAM reported
12 GB

Use cases

codingagentic

Summary

User reports Bonsai 2 27B (PTQ1_0 ternary) at 42.3 t/s plain generation and 895 t/s prompt reading on an Intel Arc B580 12GB. Setup is llama.cpp's llama-bench (tg128 and pp512) on the SYCL backend with PTQ1_0 GGUF weights. With speculation, fresh code answers reach 90 t/s, renames 365 t/s, edits 258 t/s. At 48K context new code runs 63 t/s and edits 34 t/s; at 115K, 44 and 27 t/s. The user also compares against the Vulkan backend and PrismML's own fork, and reports KL divergence of 0.00022 against reference logits.