Unknown family 0.021B
on AMD RX 9070
Summary
User reports a 21M model with a 16.8M-row lookup table (6.4B table parameters, 33M used per token) writing ~140 tok/s on an RX 9070, with the 4-bit table memory-mapped from an NVMe SSD and using 0.4 GB of VRAM.
The model matches a 114M dense model trained on the same 500M Wikipedia tokens. Reading long prompts from the SSD is slow because every missed row costs a 4 KB page.
Triton kernels run unchanged on the Radeon, an MI350X and H100/H200. Bolting a table onto Qwen3.5-0.8B did not help. Caveats: tiny model, one seed for the big runs, fluent but factually made-up text.