arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.12401cs.LG
收藏

图神经网络增强预训练用于百万级异构图

Enhanced Pre-training of Graph Neural Networks for Million-Scale Heterogeneous Graphs

City University of Hong Kong(香港城市大学);China Academy of Industrial Internet(中国工业互联网研究院)

Shengyin Sun, Chen Ma, Jiehao Chen

AI总结:

本文提出一个针对百万级异构图的GNN增强预训练框架,通过结构感知和语义感知任务解决语义不匹配,提升模型可迁移性,实验证明优于现有方法。

摘要:

近年来,图神经网络(GNNs)促进了图数据挖掘的发展。然而,训练GNNs需要足够的带标签的任务特定数据,这既昂贵有时又不可用。为了减少对标注数据的依赖,最近的研究提出以自监督方式预训练GNNs,然后将预训练的GNNs应用于标注数据有限的下游任务。然而,大多数现有方法仅针对同构图设计(现实世界的图大多是异构的),并且未考虑语义不匹配(原始数据与包含更多可迁移语义信息的理想数据之间的语义差异)。在本文中,我们提出了一个有效的框架,用于在大规模异构图上预训练GNNs。我们首先设计了一个结构感知的预训练任务,旨在捕获异构图中的结构属性。然后,我们设计了一个语义感知的预训练任务来解决不匹配问题。具体来说,我们构建了一个由语义邻居组成的扰动子空间,以帮助处理语义不匹配。语义邻居使模型更关注语义空间中的一般知识,这反过来有助于模型学习具有更好可迁移性的知识。最后,在真实世界的大规模异构图上进行了大量实验,以证明所提出的方法相对于最先进的基线方法的优越性。代码可在 https://github.com/sunshy-1/PHE 获取。

英文摘要:

In recent years, graph neural networks (GNNs) have facilitated the development of graph data mining. However, training GNNs requires sufficient labeled task-specific data, which is expensive and sometimes unavailable. To be less dependent on labeled data, recent studies propose to pre-train GNNs in a self-supervised manner and then apply the pre-trained GNNs to downstream tasks with limited labeled data. However, most existing methods are designed solely for homogeneous graphs (real-world graphs are mostly heterogeneous) and do not consider semantic mismatch (the semantic difference between the original data and the ideal data containing more transferable semantic information). In this paper, we propose an effective framework to pre-train GNNs on the large-scale heterogeneous graph. We first design a structure-aware pre-training task, which aims to capture structural properties in heterogeneous graphs. Then, we design a semantic-aware pre-training task to tackle the mismatch. Specifically, we construct a perturbation subspace composed of semantic neighbors to help deal with the semantic mismatch. Semantic neighbors make the model focus more on the general knowledge in the semantic space, which in turn assists the model in learning knowledge with better transferability. Finally, extensive experiments are conducted on real-world large-scale heterogeneous graphs to demonstrate the superiority of the proposed method over state-of-the-art baselines. Code available at https://github.com/sunshy-1/PHE.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑