重新发现哈希随机投影以实现上下文句子嵌入的高效量化
Rediscovering Hashed Random Projections for Efficient Quantization of Contextualized Sentence Embeddings
AI总结：
提出一种基于随机超平面投影和二进制量化的方法，将预计算上下文句子嵌入的内存占用减少高达98.96%，并在英德句子分类任务中保留94%至99%的浮点性能。
摘要：
由于计算限制，边缘设备上的训练和推理通常需要高效的设置。虽然预计算数据表示并将其缓存在服务器上可以缓解大量的边缘设备计算，但这带来了两个挑战。首先，服务器上所需的存储量随实例数量线性扩展。其次，将大量数据发送到边缘设备所需的带宽。为了减少预计算数据表示的内存占用，我们提出了一种简单而有效的方法，该方法使用随机初始化的超平面投影。为了进一步将其大小减少高达98.96%，我们将得到的浮点表示量化为二进制向量。尽管大小大幅减小，我们表明这些嵌入在跨各种英语和德语句子分类任务中训练模型时仍然有效，保留了其浮点表示94%--99%的性能。
英文摘要：
Training and inference on edge devices often requires an efficient setup due to computational limitations. While pre-computing data representations and caching them on a server can mitigate extensive edge device computation, this leads to two challenges. First, the amount of storage required on the server that scales linearly with the number of instances. Second, the bandwidth required to send extensively large amounts of data to an edge device. To reduce the memory footprint of pre-computed data representations, we propose a simple, yet effective approach that uses randomly initialized hyperplane projections. To further reduce their size by up to 98.96%, we quantize the resulting floating-point representations into binary vectors. Despite the greatly reduced size, we show that the embeddings remain effective for training models across various English and German sentence classification tasks that retain 94%--99% of their floating-point.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
