差距去哪了?重新评估长程图基准
Where Did the Gap Go? Reassessing the Long-Range Graph Benchmark
AI总结:
本文重新评估长程图基准,发现图变换器与消息传递GNN的性能差距因超参数欠佳被高估,优化后差距消失,并指出特征归一化和链接预测指标问题,旨在提升实证严谨性。
摘要:
最近的长程图基准(LRGB,Dwivedi 等人,2022)引入了一组强烈依赖于顶点之间长程相互作用的图学习任务。经验证据表明,在这些任务上,图变换器显著优于消息传递图神经网络(MPGNNs)。在本文中,我们仔细重新评估了LRGB上的多个MPGNN基线以及图变换器GPS(Rampášek 等人,2022)。通过严格的实证分析,我们证明了所报告的性能差距因超参数选择欠佳而被高估。值得注意的是,在多个数据集上,经过基本的超参数优化后,性能差距完全消失。此外,我们讨论了LRGB视觉数据集缺乏特征归一化的影响,并指出了LRGB链接预测指标的一个虚假实现。我们论文的主要目标是在图机器学习社区内建立更高的实证严谨性标准。
英文摘要:
The recent Long-Range Graph Benchmark (LRGB, Dwivedi et al. 2022) introduced a set of graph learning tasks strongly dependent on long-range interaction between vertices. Empirical evidence suggests that on these tasks Graph Transformers significantly outperform Message Passing GNNs (MPGNNs). In this paper, we carefully reevaluate multiple MPGNN baselines as well as the Graph Transformer GPS (Rampášek et al. 2022) on LRGB. Through a rigorous empirical analysis, we demonstrate that the reported performance gap is overestimated due to suboptimal hyperparameter choices. It is noteworthy that across multiple datasets the performance gap completely vanishes after basic hyperparameter optimization. In addition, we discuss the impact of lacking feature normalization for LRGB's vision datasets and highlight a spurious implementation of LRGB's link prediction metric. The principal aim of our paper is to establish a higher standard of empirical rigor within the graph machine learning community.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。