Leader：为更快的单词向量序列化添加长度前缀
Leader: Prefixing a Length for Faster Word Vector Serialization
摘要：
两种相互竞争的文件格式已经成为分发预训练的词嵌入的事实标准。两者都以这种格式分发的最流行的预训练嵌入命名。GloVe格式是一种完全基于文本的格式，它具有巨大的文件大小和缓慢的读取速度，而word2vec格式是一种较小的二进制格式，它混合了单词的文本表示和向量本身的二进制表示。这两种格式都有问题，我们用一种新的格式来解决，我们称之为Leader格式。我们包含了一个字长前缀，以便在保持二进制格式所提供的较小文件大小的同时实现更快的读取。我们还创建了一个极简库，以方便阅读和编写各种单词向量格式，以及将预先训练的嵌入转换为我们新的Leader格式的工具。
英文摘要：
Two competing file formats have become the de facto standards for distributing pre-trained word embeddings. Both are named after the most popular pre-trained embeddings that are distributed in that format. The GloVe format is an entirely text based format that suffers from huge file sizes and slow reads, and the word2vec format is a smaller binary format that mixes a textual representation of words with a binary representation of the vectors themselves. Both formats have problems that we solve with a new format we call the Leader format. We include a word length prefix for faster reads while maintaining the smaller file size a binary format offers. We also created a minimalist library to facilitate the reading and writing of various word vector formats, as well as tools for converting pre-trained embeddings to our new Leader format.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
