arXiv 2309.149072025-06-03cs.LGcs.AI

面向大规模属性图节点表示学习的标签反卷积方法以对抗学习偏差

Label Deconvolution for Node Representation Learning on Large-scale Attributed Graphs against Learning Bias

MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China（脑启发智能感知与认知国家重点实验室，中国科学技术大学）；Alibaba Group（阿里巴巴集团）

Zhihao Shi, Jie Wang, Fanghua Lu, Hanzhu Chen, Defu Lian, Zheng Wang, Jieping Ye, Feng Wu

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对大规模属性图上分别训练节点编码器与图神经网络导致的学习偏差问题，本文提出标签反卷积（LD）技术，通过可扩展的GNN逆映射近似实现与联合训练等价的目标函数，并在OGB数据集上显著超越现有方法。

摘要：

属性图上的节点表示学习——其节点关联着丰富的属性（例如文本和蛋白质序列）——在许多重要的下游任务中起着至关重要的作用。为了同时编码属性和图结构，近期研究将预训练模型与图神经网络（GNNs）相结合，其中预训练模型作为节点编码器（NEs）来编码属性。由于在大规模图上联合训练大型NEs和GNNs面临严重的可扩展性问题，许多方法提出分别训练NEs和GNNs。因此，这些方法在NEs的训练阶段没有考虑GNNs中的特征卷积，导致相对于联合训练存在显著的学习偏差。为了解决这一挑战，我们提出了一种高效的标签正则化技术，即标签反卷积（Label Deconvolution, LD），通过对GNNs逆映射的一种新颖且高度可扩展的近似来缓解学习偏差。该逆映射导出的目标函数与联合训练的目标函数等价，同时能够在NEs的训练阶段有效融入GNNs以对抗学习偏差。更重要的是，我们证明了在温和假设下，LD能够收敛到联合训练的最优目标函数值。实验表明，LD在Open Graph Benchmark数据集上显著优于最先进的方法。

英文摘要：

Node representation learning on attributed graphs -- whose nodes are associated with rich attributes (e.g., texts and protein sequences) -- plays a crucial role in many important downstream tasks. To encode the attributes and graph structures simultaneously, recent studies integrate pre-trained models with graph neural networks (GNNs), where pre-trained models serve as node encoders (NEs) to encode the attributes. As jointly training large NEs and GNNs on large-scale graphs suffers from severe scalability issues, many methods propose to train NEs and GNNs separately. Consequently, they do not take feature convolutions in GNNs into consideration in the training phase of NEs, leading to a significant learning bias relative to the joint training. To address this challenge, we propose an efficient label regularization technique, namely Label Deconvolution (LD), to alleviate the learning bias by a novel and highly scalable approximation to the inverse mapping of GNNs. The inverse mapping leads to an objective function that is equivalent to that by the joint training, while it can effectively incorporate GNNs in the training phase of NEs against the learning bias. More importantly, we show that LD converges to the optimal objective function values by the joint training under mild assumptions. Experiments demonstrate LD significantly outperforms state-of-the-art methods on Open Graph Benchmark datasets.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。