链路预测中评估指标的不一致性
Inconsistency of evaluation metrics in link prediction
AI总结：
本文通过大量实验发现链路预测中不同评估指标会导致算法排名显著不一致，提出应至少同时使用AUC和AUPR、AUC-Precision或NDCG中的一个指标进行评估，为建立统一的链路预测评估标准提供了依据。
摘要：
链路预测是网络科学中一个典型且具有挑战性的问题，其目标是根据已知拓扑结构预测缺失链路、未来链路和时序链路。随着链路预测算法数量的不断增加，一个关键但此前被忽视的风险是：算法性能的评估指标通常是随意选择的。本文在数百个真实网络和25种知名算法上进行了大量实验，揭示了评估指标之间存在显著的不一致性，即不同指标可能对算法产生明显不同的排名。因此，我们得出结论：任何单一指标都无法全面或可信地评估算法性能。进一步分析表明，至少应使用两个指标：一个是接收者操作特征曲线下面积（AUC），另一个是以下三个候选指标之一，即精确率-召回率曲线下面积（AUPR）、精确率曲线下面积（AUC-Precision）和归一化折损累计增益（NDCG）。此外，由于我们已经证明了阈值依赖指标之间的本质等价性，如果在某个链路预测任务中某些特定阈值是有意义的，我们可以考虑使用任意一个带有这些阈值的阈值依赖指标。这项工作补齐了链路预测研究版图中缺失的一环，并为建立一个被广泛接受的标准或准则以选择适当的链路预测评估指标提供了起点。
英文摘要：
Link prediction is a paradigmatic and challenging problem in network science, which aims to predict missing links, future links and temporal links based on known topology. Along with the increasing number of link prediction algorithms, a critical yet previously ignored risk is that the evaluation metrics for algorithm performance are usually chosen at will. This paper implements extensive experiments on hundreds of real networks and 25 well-known algorithms, revealing significant inconsistency among evaluation metrics, namely different metrics probably produce remarkably different rankings of algorithms. Therefore, we conclude that any single metric cannot comprehensively or credibly evaluate algorithm performance. Further analysis suggests the usage of at least two metrics: one is the area under the receiver operating characteristic curve (AUC), and the other is one of the following three candidates, say the area under the precision-recall curve (AUPR), the area under the precision curve (AUC-Precision), and the normalized discounted cumulative gain (NDCG). In addition, as we have proved the essential equivalence of threshold-dependent metrics, if in a link prediction task, some specific thresholds are meaningful, we can consider any one threshold-dependent metric with those thresholds. This work completes a missing part in the landscape of link prediction, and provides a starting point toward a well-accepted criterion or standard to select proper evaluation metrics for link prediction.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
