BitNet b1.58 2B
on Unknown GPU
Summary
User reports Microsoft's BitNet b1.58-2B running at 3.2 tok/s with 2 workers (2.0 tok/s serial) on a 2018 Dell laptop with an i7-7500U, 8GB RAM and no GPU.
The stack is a from-scratch C inference engine with a 1.58-bit ternary core, weights converted to a custom 2-bit packed .aimw format at 0.87 GB. The ternary core itself measured 85.5 tok/s.
The run is CPU-only; the laptop's 940MX is deliberately unused. The user notes their own trained models are not released yet.
Compare this run
Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.