arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2503.00547cs.LG
收藏

图神经网络中的性能异质性:架构设计与预处理的启示

SiGMA: Learning under Graph-Level Heterogeneity with Gated Message-Passing and Attention

Harvard University(哈佛大学)

Raphael Pellegrin, Lukas Fesser, Melanie Weber

AI总结:

本文系统分析了图级任务中的性能异质性,发现仅凭图拓扑无法解释该现象,并利用Tree Mover's Distance建立了类间距离比与性能异质性的联系,进而提出了选择性重布线和基于图谱的深度选择启发式方法以优化GNN设计。

摘要:

图神经网络已成为图级学习(包括图分类和回归任务)最流行的架构,这些任务在生物化学和药物发现等领域中经常出现。在实践中获得良好的性能需要仔细的模型设计。由于我们对模型与数据特征之间关系的理解存在差距,这通常需要手动进行架构和超参数调整。由于输入数据的变体比节点级任务高得多,这在图级任务中尤为明显。为了努力缩小这些差距,我们从系统分析图级任务中的个体性能开始。我们的结果在消息传递和基于Transformer的架构中确立了显著的性能异质性。然后,我们研究了模型与数据特征之间的相互作用作为观察到的异质性的驱动因素。我们的结果表明,仅凭图拓扑无法解释异质性。使用Tree Mover's Distance(联合评估拓扑和特征信息),我们在图分类中建立了类间距离比与性能异质性之间的联系。这些见解激发了考虑图之间异质性的模型和数据预处理选择。我们提出了一种选择性重布线方法,仅针对那些个体性能可以从重布线中受益的图。我们进一步表明,最佳网络深度取决于图的谱,这激发了一种选择GNN层数的启发式方法。我们的实验证明了这两种设计选择在实践中的效用。

英文摘要:

In graph-level learning, graph neural networks are typically applied with a fixed computational template: the same depth, local message-passing operator, and global attention mechanism are used for every input graph. However, different graphs within the same dataset can vary substantially in prediction difficulty and may require different mixtures of local and non-local computation. We study this phenomenon as graph-level heterogeneity in both classification and regression tasks. To address it, we propose SiGMA (Sigmoid Gated Message-Passing and Attention), a gated hybrid graph neural network architecture that combines message-passing and attention heads within each layer. Inspired by the growing use of gated attention and gated computation in large language models and vision transformers, each head in SiGMA is modulated by a learned node-wise sigmoid gate, allowing the model to adaptively regulate which local or global mechanisms are active for different nodes, layers, and input graphs. The resulting architecture can emulate expressive message-passing backbones, admits near-identity computation paths that are not forced to over-smooth, and enables direct non-local communication beyond message-passing neighborhoods to mitigate over-squashing. Empirically, SiGMA achieves strong performance across graph-level benchmarks, consistently matching or improving over competitive graph transformer and hybrid baselines. Our results suggest that adapting computation to graph-level heterogeneity is a promising direction for building more flexible and effective graph learning architectures.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑