arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2611.00511cs.LG
收藏

面向异构图自监督学习的高效可扩展多视图掩码自编码器

Efficient and Scalable Multi-View Masked Autoencoder for Heterogeneous Graph Self-Supervised Learning

Beihang University(北京航空航天大学);Zhongguancun Laboratory(中关村实验室);Westlake University(西湖大学);National Computer Network Emergency Response Technical Team/Coordination Center of China(国家计算机网络应急技术处理协调中心)

Hao Geng, Deqing Wang, Lihao Ge, Chenguang Du, Fuzhen Zhuang, Bowen Yang, Yanlin Hu, Songtao Wang

AI总结:

针对异构图自监督学习现有方法的三大缺陷,提出MVMAE框架,采用多视图掩码重构策略与高效离线聚合机制,在六个含大规模图的数据集上性能优于现有基线。

摘要:

近年来,异构图上的自监督学习蓬勃发展,例如基于掩码自编码器的方法。然而,现有方法存在三个问题:构建元路径诱导子图的计算成本过高、均匀随机掩码的局限性,以及重构潜在有噪声或无信息的原始节点特征的目标欠佳。为解决这些问题,我们提出了多视图掩码自编码器(MVMAE),这是一种新颖、高效且可扩展的异构图自监督学习框架。首先,MVMAE 引入了一种新颖的多视图掩码-重构策略:不再掩码单个节点或特征,而是随机掩码整个元路径视图,并引导模型从剩余可见视图中重构其表示。该方法将学习目标转向高级语义任务,提供了更鲁棒且信息丰富的监督信号,即使在初始节点特征缺失或有缺陷时也有效。其次,为应对计算需求,MVMAE 采用了高效的离线聚合机制,预计算并复用每个元路径视图的表示,完全避免了构建元路径诱导子图的高昂成本。接下来,为融合各视图的表示,MVMAE 使用基于 Transformer 的多视图融合编码器捕捉视图间的复杂关联,并采用刻意简化的 MLP 解码器,以促使编码器学习高表达性和鲁棒性的节点表示。在六个真实世界数据集(包括三个大规模图)上开展的大量实验表明,MVMAE 始终优于最先进的自监督甚至半监督基线,证实了其卓越的有效性和可扩展性。我们的源代码发布在此 https URL。

英文摘要:

Recent years have witnessed the flourishing of self-supervised learning on heterogeneous graphs, e.g., masked autoencoder-based methods. However, existing methods suffer from three issues, including the prohibitive computational cost of constructing meta-path-induced subgraphs, the limitations of uniform random masking, and the suboptimal objective of reconstructing potentially noisy or uninformative original node features. To address these issues, we propose the Multi-View Masked Auto Encoder (MVMAE), a novel, efficient, and scalable framework for heterogeneous graph self-supervised learning. First, MVMAE introduces a novel multi-view mask-then-reconstruct strategy. Instead of masking individual nodes or features, MVMAE randomly masks entire meta-path views and guides the model to reconstruct their representations from remaining visible views. This approach shifts the learning objective to a high-level and semantic task, providing a more robust and informative supervisory signal that is effective even when initial node features are absent or flawed. Then, to handle the computational demands, MVMAE employs an efficient offline aggregation mechanism that pre-computes and reuses representations for each meta-path view, completely avoiding the costly construction of meta-path-induced subgraphs. Next, to fuse the representations of each view, MVMAE uses a Transformer-based multi-view fusion encoder to capture the complex correlations between views, and a deliberately simple MLP decoder to compel the encoder to learn highly expressive and robust node representations. Extensive experiments on six real-world datasets, including three large-scale graphs, demonstrate that MVMAE consistently outperforms state-of-the-art self-supervised and even semi-supervised baselines, confirming its superior effectiveness and scalability. Our source code is released at https://github.com/hcmdgh/MVMAE.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑