arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2104.06924cs.CL
收藏

无监督实体评价与事件显著性估计

Evaluation of Unsupervised Entity and Event Salience Estimation

Emory University(埃默里大学)

Jiaying Lu, Jinho D. Choi

摘要:

显著性估计的目的是预测文档中的术语重要性。由于现有的人类注释数据集很少,以及显着性的主观概念,以前的研究通常会生成用于评估的伪地面实况。然而,我们的调查显示,以前的工作提出的评估协议是难以复制的,从而导致一些后续研究存在。此外,评估过程是有问题的:用于实体匹配的实体链接工具是非常嘈杂的,而忽略事件评估的事件参数导致性能提升。在这项工作中,我们提出了一个轻而实用的实体和事件的显着性估计评估协议,其中包括更可靠的句法依赖分析器。此外,我们对流行的实体和事件定义标准进行了全面的分析,并提出了我们自己的显着性估计任务的定义,以减少伪地面实况生成过程中的噪声。此外,我们构建了基于依赖关系的异构图,以捕捉实体和事件的相互作用。实证结果表明,基线方法和新的GNN方法利用异构图始终优于以前的SOTA模型在所有提出的指标。

英文摘要:

Salience Estimation aims to predict term importance in documents. Due to few existing human-annotated datasets and the subjective notion of salience, previous studies typically generate pseudo-ground truth for evaluation. However, our investigation reveals that the evaluation protocol proposed by prior work is difficult to replicate, thus leading to few follow-up studies existing. Moreover, the evaluation process is problematic: the entity linking tool used for entity matching is very noisy, while the ignorance of event argument for event evaluation leads to boosted performance. In this work, we propose a light yet practical entity and event salience estimation evaluation protocol, which incorporates the more reliable syntactic dependency parser. Furthermore, we conduct a comprehensive analysis among popular entity and event definition standards, and present our own definition for the Salience Estimation task to reduce noise during the pseudo-ground truth generation process. Furthermore, we construct dependency-based heterogeneous graphs to capture the interactions of entities and events. The empirical results show that both baseline methods and the novel GNN method utilizing the heterogeneous graph consistently outperform the previous SOTA model in all proposed metrics.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑