llamaperf

BitNet b1.58 2B

on AMD Radeon Pro W7900 · AimStack

Tone: mixedAMD hardware
Nov 18, 2026
Throughput
13.7 t/s gen
Quant
ternary (AIM-W)
System RAM
8 GB

Summary

User reports BitNet b1.58 2B at 13.68 tok/s on a Radeon Pro W7900 with a self-built C inference stack, AimStack. The GPU backend loads the same 0.87GB ternary file, dequantizes weights on load and runs matmul in fp32; 256 of 256 greedy tokens matched the C engine bit-for-bit. The C engine runs the full model on a 2018 dual-core 8GB laptop at 3.20 tok/s with two workers, and its SIMD path measured 85.53 tok/s. BF16 is 2.4x faster at 32.96 tok/s but flips argmax on near-tied logits, so only FP32 passes validation. The GPU path gets the 78% memory saving but not yet a compute advantage; the UI server drives only the CPU C engine.

Compare this run

Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.

Models most run on the AMD Radeon Pro W7900

Bars are medians on one scale, 0 to 95.0 tokens/s. Press ▶ to watch text arrive at a row's speed.

Everything reported on the AMD Radeon Pro W7900 →Every BitNet b1.58 report →Estimate BitNet b1.58 on this card →