arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2004.06747cs.IRcs.CL
收藏

将文本信息性度量扩展到段落兴趣度评估(语言模型与词嵌入)

Extending Text Informativeness Measures to Passage Interestingness Evaluation (Language Model vs. Word Embedding)

Université d’Avignon et des Pays de Vaucluse(阿维尼翁大学);École Polytechnique de Montréal(蒙特利尔高等理工学院)

Carlos-Emiliano González-Gallardo, Eric SanJuan, Juan-Manuel Torres-Moreno

摘要:

用于评估自动文本摘要的标准信息性度量主要依赖于自动摘要和参考摘要之间的n-gram重叠。这些度量不同于它们使用的度量(余弦,ROUGE,Kullback-Leibler,对数相似性等)。以及他们考虑的术语包(单个词、词n-gram、实体、nuggets等)。最近的词嵌入方法提供了一个连续的替代离散的方法的基础上的存在/不存在的文本单元。信息量的措施已扩展到焦点信息检索评价涉及用户的信息需求表示的短查询。特别是对于CLEF-INEX推文上下文化的任务,推文内容被认为是查询。在本文中,我们定义的概念,概括的信息性,即信息需求是多样的,形式化为一个未知的一组隐式的查询。然后,我们研究国家的最先进的信息量的措施,以应付这种泛化的能力。最近,我们表明,在这个新的框架,标准的词嵌入优于离散措施,只有在uni-gram,但bi-gram似乎是一个关键点的兴趣评估。最后,我们证明了CLEF-INEX推文语境化2012对数相似性度量提供了最佳结果。

英文摘要:

Standard informativeness measures used to evaluate Automatic Text Summarization mostly rely on n-gram overlapping between the automatic summary and the reference summaries. These measures differ from the metric they use (cosine, ROUGE, Kullback-Leibler, Logarithm Similarity, etc.) and the bag of terms they consider (single words, word n-grams, entities, nuggets, etc.). Recent word embedding approaches offer a continuous alternative to discrete approaches based on the presence/absence of a text unit. Informativeness measures have been extended to Focus Information Retrieval evaluation involving a user's information need represented by short queries. In particular for the task of CLEF-INEX Tweet Contextualization, tweet contents have been considered as queries. In this paper we define the concept of Interestingness as a generalization of Informativeness, whereby the information need is diverse and formalized as an unknown set of implicit queries. We then study the ability of state of the art Informativeness measures to cope with this generalization. Lately we show that with this new framework, standard word embeddings outperforms discrete measures only on uni-grams, however bi-grams seems to be a key point of interestingness evaluation. Lastly we prove that the CLEF-INEX Tweet Contextualization 2012 Logarithm Similarity measure provides best results.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑