EFloat：用于压缩向量嵌入模型的熵编码浮点格式
EFloat: Entropy-coded Floating Point Format for Compressing Vector Embedding Models
摘要：
在一大类深度学习模型中，包括矢量嵌入模型，如单词和数据库嵌入，我们观察到浮点指数值聚集在一些唯一值周围，允许基于熵的数据压缩。熵编码用可变长度代码压缩固定长度的值，用更少的比特编码最可能的值。我们提出了EFloat压缩的浮点数格式，它在指数和有效位字段之间使用可变字段边界。EFloat对指数值和符号使用熵编码来最小化指数和符号字段的平均宽度，同时保持原始FP32指数范围不变。保存的位成为有效位字段的一部分，与其他精度降低的浮点格式相比，EFloat的数值精度平均提高了4.3位。EFloat使8位甚至更小的浮点数变得实用，而不会牺牲32位浮点表示的指数范围。我们目前使用EFloat格式来节省大型向量嵌入模型（如用于数据库嵌入的模型）的内存容量和带宽消耗。使用RMS误差作为度量，我们证明了EFloat比其他浮点格式具有更高的精度。具有12位预算的EF12格式比16位BFloat16具有更少的端到端应用程序错误。对于不同的嵌入模型集，具有16位预算的EF16的RMS误差比BF 16的RMS误差小17到35倍。当进行相似性和不相似性查询时，使用NDCG排名指标，EFloat将先前浮点表示的结果质量与更大的位预算相匹配。
英文摘要：
In a large class of deep learning models, including vector embedding models such as word and database embeddings, we observe that floating point exponent values cluster around a few unique values, permitting entropy based data compression. Entropy coding compresses fixed-length values with variable-length codes, encoding most probable values with fewer bits. We propose the EFloat compressed floating point number format that uses a variable field boundary between the exponent and significand fields. EFloat uses entropy coding on exponent values and signs to minimize the average width of the exponent and sign fields, while preserving the original FP32 exponent range unchanged. Saved bits become part of the significand field increasing the EFloat numeric precision by 4.3 bits on average compared to other reduced-precision floating point formats. EFloat makes 8-bit and even smaller floats practical without sacrificing the exponent range of a 32-bit floating point representation. We currently use the EFloat format for saving memory capacity and bandwidth consumption of large vector embedding models such as those used for database embeddings. Using the RMS error as metric, we demonstrate that EFloat provides higher accuracy than other floating point formats with equal bit budget. The EF12 format with 12-bit budget has less end-to-end application error than the 16-bit BFloat16. EF16 with 16-bit budget has an RMS-error 17 to 35 times less than BF16 RMS-error for a diverse set of embedding models. When making similarity and dissimilarity queries, using the NDCG ranking metric, EFloat matches the result quality of prior floating point representations with larger bit budgets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
