arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1801.09536cs.CL
收藏

词嵌入评价方法综述

A Survey of Word Embeddings Evaluation Methods

Institute for System Analysis of Russian Academy of Sciences (ISA RAS)(俄罗斯科学院系统分析研究所(ISA RAS));The National Research University Higher School of Economics(俄罗斯国家研究型高等经济大学)

Amir Bakarov

摘要:

词嵌入是能够捕获词汇语义并在自然语言语料库上训练的实值词表示。近年来,提出这些表示的模型已经流行起来,但最适当的评估方法的问题仍然是开放的。本文综述了词嵌入评价领域的研究现状,指出了存在的主要问题,提出了评价方法的类型,总结了16种内在评价方法和12种外在评价方法。我描述了广泛使用的和实验性的方法,系统化有关评估数据集的信息,并讨论了一些关键挑战。

英文摘要:

Word embeddings are real-valued word representations able to capture lexical semantics and trained on natural language corpora. Models proposing these representations have gained popularity in the recent years, but the issue of the most adequate evaluation method still remains open. This paper presents an extensive overview of the field of word embeddings evaluation, highlighting main problems and proposing a typology of approaches to evaluation, summarizing 16 intrinsic methods and 12 extrinsic methods. I describe both widely-used and experimental methods, systematize information about evaluation datasets and discuss some key challenges.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑