arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1606.00819cs.CL
收藏

使用窗口采样和负采样的矩阵分解改进单词表示

Matrix Factorization using Window Sampling and Negative Sampling for Improved Word Representations

Institute of Informatics(信息学研究所);Universidade Federal do Rio Grande do Sul(南里奥格兰德联邦大学)

Alexandre Salle, Marco Idiart, Aline Villavicencio

摘要:

在本文中,我们提出了LexVec,一种新的方法,用于生成分布式单词表示,使用低秩,加权因式分解的正逐点互信息矩阵通过随机梯度下降,采用加权方案,分配较重的惩罚频繁同现的错误,同时仍然占负同现。对单词相似性和类比任务的评估表明,LexVec在许多这些任务上匹配并经常优于最先进的方法。

英文摘要:

In this paper, we propose LexVec, a new method for generating distributed word representations that uses low-rank, weighted factorization of the Positive Point-wise Mutual Information matrix via stochastic gradient descent, employing a weighting scheme that assigns heavier penalties for errors on frequent co-occurrences while still accounting for negative co-occurrence. Evaluation on word similarity and analogy tasks shows that LexVec matches and often outperforms state-of-the-art methods on many of these tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑