arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1806.01330cs.CLstat.ML
收藏

封闭式词嵌入对齐

Closed Form Word Embedding Alignment

School of Computing(计算学院)

Sunipa Dev, Safia Hassan, Jeff M. Phillips

摘要:

我们开发了一系列技术来对齐来自不同源数据集或使用不同机制(例如,GloVe或word2vec)。我们的方法很简单,并且有一个封闭的形式来优化旋转,平移和缩放,以最大限度地减少均方根误差或最大限度地提高相同词汇的两个嵌入到同一维空间之间的平均余弦相似性。我们的方法扩展了称为绝对方向的方法,这种方法在三维中对齐对象时很流行,并推广了Smith等人的方法(ICLR 2017)。我们证明了新的最佳缩放和最大限度地提高余弦相似性的结果。然后,我们演示了如何评估来自不同来源或机制的嵌入的相似性,以及在嵌入中保留某些属性(如同义词和类比),并且可以通过简单地对齐和平均嵌入集合来增强。

英文摘要:

We develop a family of techniques to align word embeddings which are derived from different source datasets or created using different mechanisms (e.g., GloVe or word2vec). Our methods are simple and have a closed form to optimally rotate, translate, and scale to minimize root mean squared errors or maximize the average cosine similarity between two embeddings of the same vocabulary into the same dimensional space. Our methods extend approaches known as Absolute Orientation, which are popular for aligning objects in three-dimensions, and generalize an approach by Smith etal (ICLR 2017). We prove new results for optimal scaling and for maximizing cosine similarity. Then we demonstrate how to evaluate the similarity of embeddings from different sources or mechanisms, and that certain properties like synonyms and analogies are preserved across the embeddings and can be enhanced by simply aligning and averaging ensembles of embeddings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑