arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2003.04628cs.IR
收藏

关键词提取模型的大规模评估

Large-Scale Evaluation of Keyphrase Extraction Models

Ygor Gallina, Florian Boudin, Béatrice Daille

摘要:

关键词提取模型通常在不同的,不直接可比的实验设置下进行评估。因此,目前尚不清楚所提出的模型实际表现如何,以及它们之间的比较情况。在这项工作中,我们通过对涉及来自各种来源和领域的多个基准数据集的最先进的关键词提取模型进行系统的大规模分析来解决这个问题。我们的主要结果表明,最先进的模型实际上仍然受到一些数据集上的简单基线的挑战。我们还提出了关于使用作者或读者指定的关键词作为黄金标准代理的影响的新见解,并给出了强基线和可靠基准数据集的建议。

英文摘要:

Keyphrase extraction models are usually evaluated under different, not directly comparable, experimental setups. As a result, it remains unclear how well proposed models actually perform, and how they compare to each other. In this work, we address this issue by presenting a systematic large-scale analysis of state-of-the-art keyphrase extraction models involving multiple benchmark datasets from various sources and domains. Our main results reveal that state-of-the-art models are in fact still challenged by simple baselines on some datasets. We also present new insights about the impact of using author- or reader-assigned keyphrases as a proxy for gold standard, and give recommendations for strong baselines and reliable benchmark datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑