arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2304.04779cs.LG
收藏

GraphMAE2：一种解码增强的掩码自监督图学习器

GraphMAE2: A Decoding-Enhanced Masked Self-Supervised Graph Learner

Tsinghua University（清华大学）；Beijing Institute of Technology（北京理工大学）；Bosch Center for Artificial Intelligence（博世人工智能中心）

Zhenyu Hou, Yufei He, Yukuo Cen, Xiao Liu, Yuxiao Dong, Evgeny Kharlamov, Jie Tang

备注：
Accepted to WWW'23
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

GraphMAE2通过多视图随机重新掩码解码和潜在表示预测对特征重建施加正则化，解决了掩码图自编码器对输入特征判别能力的依赖问题，并在多个大规模图数据集上取得最优性能。

摘要：

图自监督学习（SSL），包括对比式和生成式方法，为解决现实世界图数据中标签稀缺这一根本挑战提供了巨大潜力。在两类图SSL技术中，掩码图自编码器（例如GraphMAE）——一种生成式方法——近期取得了令人鼓舞的结果。其核心思想是利用自编码器架构重建从输入中随机掩码的节点特征（或结构）。然而，掩码特征重建的性能天然依赖于输入特征的判别能力，并且通常容易受到特征中扰动的干扰。在本文中，我们提出了一种掩码自监督学习框架GraphMAE2，旨在克服这一问题。其思想是对图SSL的特征重建施加正则化。具体而言，我们设计了多视图随机重新掩码解码和潜在表示预测两种策略来正则化特征重建。多视图随机重新掩码解码是在特征空间中为重建引入随机性，而潜在表示预测则是在嵌入空间中强制进行重建。大量实验表明，GraphMAE2能够在各种公开数据集上持续取得最优结果，包括在拥有1.11亿节点和16亿条边的ogbn-Papers100M数据集上，相较于最先进的基线方法至少提升了2.45%。

英文摘要：

Graph self-supervised learning (SSL), including contrastive and generative approaches, offers great potential to address the fundamental challenge of label scarcity in real-world graph data. Among both sets of graph SSL techniques, the masked graph autoencoders (e.g., GraphMAE)--one type of generative method--have recently produced promising results. The idea behind this is to reconstruct the node features (or structures)--that are randomly masked from the input--with the autoencoder architecture. However, the performance of masked feature reconstruction naturally relies on the discriminability of the input features and is usually vulnerable to disturbance in the features. In this paper, we present a masked self-supervised learning framework GraphMAE2 with the goal of overcoming this issue. The idea is to impose regularization on feature reconstruction for graph SSL. Specifically, we design the strategies of multi-view random re-mask decoding and latent representation prediction to regularize the feature reconstruction. The multi-view random re-mask decoding is to introduce randomness into reconstruction in the feature space, while the latent representation prediction is to enforce the reconstruction in the embedding space. Extensive experiments show that GraphMAE2 can consistently generate top results on various public datasets, including at least 2.45% improvements over state-of-the-art baselines on ogbn-Papers100M with 111M nodes and 1.6B edges.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑