TurboQuant: Reducing LLM Memory Usage With Vector Quantization

TurboQuant is a technique that uses vector quantization to reduce the memory footprint of large language models, making them more efficient to run on limited hardware.