arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2003.08001cs.AIcs.CL
收藏

知识图谱补全方法的现实性再评价:一项实验研究

Realistic Re-evaluation of Knowledge Graph Completion Methods: An Experimental Study

University of Texas at Arlington(德克萨斯大学阿灵顿分校);Nanjing University(南京大学)

Farahnaz Akrami, Mohammed Samiul Saeef, Qingheng Zhang, Wei Hu, Chengkai Li

摘要:

在将嵌入模型用于知识图谱完成的活跃研究领域,特别是用于链接预测的任务中,大多数先前的研究使用两个基准数据集FB 15 k和WN 18来评估这样的模型。此类研究中的这些数据集和其他数据集中的大多数三元组属于反向和重复关系,由于语义重复、相关性或数据不完整性,这些关系表现出高数据冗余。这是一种过度数据泄漏的情况--当模型需要应用于实际预测时,使用无法使用的特征来训练模型。也有笛卡尔积关系,对于这些笛卡尔积关系,由适用的主体和客体的笛卡尔积形成的每个三元组都是真事实。对上述关系的链接预测是容易的,并且可以使用直接的规则而不是复杂的嵌入模型以甚至更好的精度来实现。这些模型的一个更根本的缺陷是,给定这些数据的链接预测场景在现实世界中是不存在的。本文是第一个系统的研究,其主要目标是评估嵌入模型的真实有效性时,不切实际的三元组被删除。我们的实验结果表明,这些模型远不如我们过去所感知的准确。他们的准确性差,使链接预测的任务,没有真正有效的自动化解决方案。因此,我们呼吁重新研究可能的有效办法。

英文摘要:

In the active research area of employing embedding models for knowledge graph completion, particularly for the task of link prediction, most prior studies used two benchmark datasets FB15k and WN18 in evaluating such models. Most triples in these and other datasets in such studies belong to reverse and duplicate relations which exhibit high data redundancy due to semantic duplication, correlation or data incompleteness. This is a case of excessive data leakage---a model is trained using features that otherwise would not be available when the model needs to be applied for real prediction. There are also Cartesian product relations for which every triple formed by the Cartesian product of applicable subjects and objects is a true fact. Link prediction on the aforementioned relations is easy and can be achieved with even better accuracy using straightforward rules instead of sophisticated embedding models. A more fundamental defect of these models is that the link prediction scenario, given such data, is non-existent in the real-world. This paper is the first systematic study with the main objective of assessing the true effectiveness of embedding models when the unrealistic triples are removed. Our experiment results show these models are much less accurate than what we used to perceive. Their poor accuracy renders link prediction a task without truly effective automated solution. Hence, we call for re-investigation of possible effective approaches.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑