BitNet b1.58 2B
AMD Radeon Pro W7900 · AimStack
- generation:
- 13.7 tokens/s
- quant:
- ternary (AIM-W)
User reports BitNet b1.58 2B at 13.68 tok/s on a Radeon Pro W7900 with a self-built C inference stack, AimStack. The GPU backend loads the same 0.87GB ternary file, dequantizes weights on load and runs matmul in fp32; 256 of 256 greedy tokens matched the C engine bit-for-bit. The C engine runs the full model on a 2018 dual-core 8GB laptop at 3.20 tok/s with two workers, and its SIMD path measured 85.53 tok/s. BF16 is 2.4x faster at 32.96 tok/s but flips argmax on near-tied logits, so only FP32 passes validation. The GPU path gets the 78% memory saving but not yet a compute advantage; the UI server drives only the CPU C engine.