arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2203.10135cs.LGcs.AI
收藏

用于设备上推理的学习压缩嵌入

Learning Compressed Embeddings for On-Device Inference

Apple(苹果公司);JP Morgan Chase(摩根大通)

Niketan Pansare, Jay Katukuri, Aditya Arora, Frank Cipollone, Riyaaz Shaik, Noyan Tokgozoglu, Chandru Venkataraman

摘要:

在深度学习中,嵌入被广泛用于表示分类实体,如单词、应用程序和电影。嵌入层将每个实体映射到一个唯一的向量,使层的内存需求与实体的数量成正比。在推荐领域,一个给定的类别可以有数十万个实体,其嵌入层可以占用千兆字节的内存。这些网络的规模使得它们难以在资源受限的环境中部署。在本文中,我们提出了一种新的方法来减少嵌入表的大小,同时仍然映射每个实体到自己的唯一嵌入。我们不是维护完整的嵌入表,而是使用两个单独的嵌入表“动态”构建每个实体的嵌入。第一个表使用哈希来强制多个实体共享一个嵌入。第二个表包含每个实体的一个可训练权重,允许模型区分共享相同嵌入的实体。由于这两个表是联合训练的,因此网络能够学习每个实体的唯一嵌入,帮助它保持与具有未压缩嵌入表的模型类似的区分能力。我们称这种方法为MEmCom(多重嵌入压缩)。我们比较了国家的最先进的模型压缩技术的多个问题类,包括分类和排名。在四个流行的推荐系统数据集上,MEmCom在nDCG中有4%的相对损失,同时将我们的推荐模型的输入嵌入大小压缩了16倍,4倍,12倍和40倍。MEmCom的性能优于最先进的技术,在相应的压缩比下,这些技术在nDCG中实现了16%、6%、10%和8%的相对损失。此外,MEmCom能够将RankNet排名模型在数百万用户与游戏交互的数据集上压缩32倍,同时在nDCG中仅产生1%的相对损失。

英文摘要:

In deep learning, embeddings are widely used to represent categorical entities such as words, apps, and movies. An embedding layer maps each entity to a unique vector, causing the layer's memory requirement to be proportional to the number of entities. In the recommendation domain, a given category can have hundreds of thousands of entities, and its embedding layer can take gigabytes of memory. The scale of these networks makes them difficult to deploy in resource constrained environments. In this paper, we propose a novel approach for reducing the size of an embedding table while still mapping each entity to its own unique embedding. Rather than maintaining the full embedding table, we construct each entity's embedding "on the fly" using two separate embedding tables. The first table employs hashing to force multiple entities to share an embedding. The second table contains one trainable weight per entity, allowing the model to distinguish between entities sharing the same embedding. Since these two tables are trained jointly, the network is able to learn a unique embedding per entity, helping it maintain a discriminative capability similar to a model with an uncompressed embedding table. We call this approach MEmCom (Multi-Embedding Compression). We compare with state-of-the-art model compression techniques for multiple problem classes including classification and ranking. On four popular recommender system datasets, MEmCom had a 4% relative loss in nDCG while compressing the input embedding sizes of our recommendation models by 16x, 4x, 12x, and 40x. MEmCom outperforms the state-of-the-art techniques, which achieved 16%, 6%, 10%, and 8% relative loss in nDCG at the respective compression ratios. Additionally, MEmCom is able to compress the RankNet ranking model by 32x on a dataset with millions of users' interactions with games while incurring only a 1% relative loss in nDCG.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑