arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2012.00728cs.CL
收藏

对偶词嵌入空间模型的内在分析

Intrinsic analysis for dual word embedding space models

Tata Consultancy Service(塔塔咨询服务公司)

Mohit Mayank

摘要:

最近的词嵌入技术在连续向量空间中表示词,远离过去的原子和稀疏表示。每种这样的技术可以进一步基于超参数的不同设置创建多种嵌入,所述超参数如嵌入维度大小、上下文窗口大小和训练方法。当我们特别考虑生成不是一个而是两个词嵌入作为输出的对偶嵌入空间技术时,出现了另一种变体。这就引出了一个有趣的问题--“是否存在一种或两种词嵌入的组合,哪一种更适合特定的任务?".本文试图通过考虑所有这些变化来回答这个问题。在这里,我们比较了属于两种不同方法的两种经典嵌入方法-基于窗口的Word 2 Vec和基于计数的Glove。为了在考虑所有变化后进行广泛的评估,将总共84个不同的模型与由9个开源语言学数据集组成的语义,关联和类比评估任务进行了比较。最终的Word 2 vec报告展示了3个任务中有2个任务的非默认模型的首选项。在Glove的情况下,非默认模型在所有3个评估任务中表现出色。

英文摘要:

Recent word embeddings techniques represent words in a continuous vector space, moving away from the atomic and sparse representations of the past. Each such technique can further create multiple varieties of embeddings based on different settings of hyper-parameters like embedding dimension size, context window size and training method. One additional variety appears when we especially consider the Dual embedding space techniques which generate not one but two-word embeddings as output. This gives rise to an interesting question - "is there one or a combination of the two word embeddings variety, which works better for a specific task?". This paper tries to answer this question by considering all of these variations. Herein, we compare two classical embedding methods belonging to two different methodologies - Word2Vec from window-based and Glove from count-based. For an extensive evaluation after considering all variations, a total of 84 different models were compared against semantic, association and analogy evaluations tasks which are made up of 9 open-source linguistics datasets. The final Word2vec reports showcase the preference of non-default model for 2 out of 3 tasks. In case of Glove, non-default models outperform in all 3 evaluation tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑