模型有多尖锐和偏见鲁棒？知识图谱补全的双评价视角
How Sharp and Bias-Robust is a Model? Dual Evaluation Perspectives on Knowledge Graph Completion
AI总结：
本文提出PROBE框架，通过预测尖锐度和流行度-偏见鲁棒性双视角，改进知识图谱补全的评估方法，提供更准确的模型评估结果。
摘要：
知识图谱补全（KGC）旨在从观察到的KG中预测缺失的事实。尽管已经研究了多种KGC模型，但KGC的评估仍然处于探索阶段。在本文中，我们发现现有指标忽略了KGC评估的两个关键视角：（A1）预测尖锐度——评估单个预测严格程度的程度，以及（A2）流行度-偏见鲁棒性——预测低流行度实体的能力。为了反映这两个视角，我们提出了一种新的评估框架（PROBE），它由一个排名转换器（RT）组成，该转换器根据所需程度的预测尖锐度来估计每个预测的分数，以及一个排名聚合器（RA），该聚合器以流行度意识的方式聚合所有分数。在真实世界的KG上的实验表明，现有指标倾向于过高或过低地估计KGC模型的准确性，而PROBE则提供了对KGC模型的全面理解和可靠的评估结果。
英文摘要：
Knowledge graph completion (KGC) aims to predict missing facts from the observed KG. While a number of KGC models have been studied, the evaluation of KGC still remain underexplored. In this paper, we observe that existing metrics overlook two key perspectives for KGC evaluation: (A1) predictive sharpness -- the degree of strictness in evaluating an individual prediction, and (A2) popularity-bias robustness -- the ability to predict low-popularity entities. Toward reflecting both perspectives, we propose a novel evaluation framework (PROBE), which consists of a rank transformer (RT) estimating the score of each prediction based on a required level of predictive sharpness and a rank aggregator (RA) aggregating all the scores in a popularity-aware manner. Experiments on real-world KGs reveal that existing metrics tend to over- or under-estimate the accuracy of KGC models, whereas PROBE yields a comprehensive understanding of KGC models and reliable evaluation results.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
