llamaperf

Bonsai 2 27B

on Intel Arc B580 12GB · llama.cpp · 128,000 ctx

Tone: positiveIntel hardware
Oct 12, 2026
Throughput
48.6 t/s gen · 1480.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
ternary (GGUF)
KV cache
q4_0
MTP (Multi-Token Prediction)
on
VRAM reported
12 GB

Use cases

coding

Summary

User reports ternary Bonsai 2 27B at 48.6 t/s plain decode on an Intel Arc B580 12GB. Setup is a custom llama.cpp SYCL branch with ternary weights, q4_0 KV cache and 128K context, with no speculation for this figure. With MTP, n-gram speculation and a trimmed draft head the model reaches 85 t/s writing code, 79 t/s in chat and 89 t/s at 115K tokens of history, with prompt reading at ~1,480 t/s. The user's own draft head was used for the video, so public results may be a few percent lower.