链接预测中的数值字面量:对模型和数据集的批判性考察
Numerical Literals in Link Prediction: A Critical Examination of Models and Datasets
AI总结:
本文提出一种评估结合数值字面量的链接预测模型的方法论,通过构建合成数据集与消融策略,发现许多模型未充分利用字面量信息,揭示了现有基准数据集的不足及更广泛评估的必要性。
摘要:
链接预测(LP)是知识图谱(KGs)上的一项基本任务,传统上侧重于使用和预测实体间的关系。文本实体描述已被证明具有价值,但结合数值字面量的模型在现有基准数据集上仅显示出微小的改进。目前尚不清楚模型是确实更擅长使用数值字面量,还是更善于利用图结构。这引发了人们对这些方法的有效性以及现有基准数据集适用性的怀疑。我们提出了一种评估结合数值字面量的LP模型的方法论。我们提出了i)一个新的合成数据集,以更好地理解这些模型使用数值字面量的程度;ii)数据集消融策略,以调查现有数据集的潜在困难。我们识别出一个普遍趋势:许多模型未充分利用字面量信息,并可能依赖额外参数来获取性能提升。我们的调查强调,在发布新模型和数据集时需要进行更广泛的评估。
英文摘要:
Link Prediction(LP) is an essential task over Knowledge Graphs(KGs), traditionally focussed on using and predicting the relations between entities. Textual entity descriptions have already been shown to be valuable, but models that incorporate numerical literals have shown minor improvements on existing benchmark datasets. It is unclear whether a model is actually better in using numerical literals, or better capable of utilizing the graph structure. This raises doubts about the effectiveness of these methods and about the suitability of the existing benchmark datasets. We propose a methodology to evaluate LP models that incorporate numerical literals. We propose i) a new synthetic dataset to better understand how well these models use numerical literals and ii) dataset ablations strategies to investigate potential difficulties with the existing datasets. We identify a prevalent trend: many models underutilize literal information and potentially rely on additional parameters for performance gains. Our investigation highlights the need for more extensive evaluations when releasing new models and datasets.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。