真实网络链路预测中评估指标区分能力的量化研究
Quantifying discriminability of evaluation metrics in link prediction for real networks
AI总结:
本文提出一种量化评估指标区分能力的新度量,基于131个真实网络和20种算法比较8种指标,发现H-measure和AUC区分能力最强,其次为NDCG,为链路预测评估指标选择提供依据。
摘要:
链路预测是网络科学中成果最丰富的分支之一,旨在预测本应存在但尚未被观测到的链接,或预测在网络演化过程中将会出现的链接。近二十年来,链路预测领域积累了大量的研究成果,涵盖众多算法和多样化的应用。对于任何算法,都需要一个或多个评估指标来评价其性能。由于使用不同的评估指标会对算法性能给出不同的评价,因此如何选择合适的评估指标是链路预测中的一个基本问题。为了解决这一问题,我们提出了一种新的度量,用于在给定真实网络和算法的情况下量化任意评估指标的区分能力。基于131个真实网络和20个代表性算法,我们系统比较了8种评估指标的区分能力,并证明H-measure和受试者工作特征曲线下面积(AUC)表现出最强的区分能力,其次是归一化折损累计增益(NDCG)。我们的发现对于不同领域的网络和不同类型的算法都具有稳健性。该研究为评估指标的选择提供了见解,并可能进一步有助于规范链路预测算法的评估流程。
英文摘要:
Link prediction is one of the most productive branches in network science, aiming to predict links that would have existed but have not yet been observed, or links that will appear during the evolution of the network. Over nearly two decades, the field of link prediction has amassed a substantial body of research, encompassing a plethora of algorithms and diverse applications. For any algorithm, one or more evaluation metrics are required to assess its performance. Because using different evaluation metrics can provide different assessments of the algorithm performance, how to select appropriate evaluation metrics is a fundamental issue in link prediction. To address this issue, we propose a novel measure that quantifiers the discriminability of any evaluation metric given a real network and an algorithm. Based on 131 real networks and 20 representative algorithms, we systematically compare the discriminabilities of eight evaluation metrics, and demonstrate that H-measure and Area Under the ROC Curve (AUC) exhibit the strongest discriminabilities, followed by Normalized Discounted Cumulative Gain (NDCG). Our finding is robust for networks in different domains and algorithms of different types. This study provides insights into the selection of evaluation metrics, which may further contribute to standardizing the evaluating process of link prediction algorithms.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。