arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1402.3722cs.CLcs.LGstat.ML
收藏

word2vec解释:导出Mikolov等人。负采样词嵌入法

word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method

Yoav Goldberg, Omer Levy

摘要:

Tomas Mikolov及其同事的word2vec软件(code.google.com/p/word2vec/)最近获得了很大的吸引力,并提供了最先进的词嵌入。该软件背后的学习模型在两篇研究论文中进行了描述。我们发现这些论文中对模型的描述有些晦涩难懂。虽然动机和演示对于神经网络语言建模人群来说可能是显而易见的,但我们不得不努力找出方程式背后的基本原理。本文试图解释Tomas Mikolov,Ilya Sutskever,Kai Chen,Greg Corrado和Jeffrey Dean的“Distributed Representations of Words and Phrases and their Compositionality”中的公式(4)(负采样)。

英文摘要:

The word2vec software of Tomas Mikolov and colleagues (https://code.google.com/p/word2vec/ ) has gained a lot of traction lately, and provides state-of-the-art word embeddings. The learning models behind the software are described in two research papers. We found the description of the models in these papers to be somewhat cryptic and hard to follow. While the motivations and presentation may be obvious to the neural-networks language-modeling crowd, we had to struggle quite a bit to figure out the rationale behind the equations. This note is an attempt to explain equation (4) (negative sampling) in "Distributed Representations of Words and Phrases and their Compositionality" by Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado and Jeffrey Dean.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑