arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.05087cs.LG
收藏

用于图自监督学习的节点与邻居引导潜变量

Bootstrap Latents of Nodes and Neighbors for Graph Self-Supervised Learning

Nanjing University(南京大学);Tsinghua University(清华大学)

Yunhui Liu, Huaisong Zhang, Tieke He, Tao Zheng, Jianhua Zhao

AI总结:

针对 BGRL 忽略图同配性的问题,本文利用节点—邻居对扩展正样本集,并以交叉注意力加权邻居支持度,在五个基准和三项任务上达到最先进性能。

摘要:

对比学习是图自监督学习中的一种重要范式。然而,它需要负样本来防止模型坍塌并学习具有判别性的表示。这些负样本不可避免地导致高昂的计算和内存开销以及类别碰撞,从而损害表示学习。近期研究表明,无需负样本的方法可以取得具有竞争力的性能并提升可扩展性,bootstrapped graph latents(BGRL,引导图潜变量)即为一例。但 BGRL 忽略了图固有的同配性,而这种同配性能够为潜在正样本对提供有价值的洞察。我们的动机来自如下观察:巧妙地引入少量真实正样本对可显著改进 BGRL。尽管在自监督设置下无法在没有标签的情况下获得真实正样本对,但图中的边可以反映含噪正样本对,即相邻节点通常共享相同标签。因此,我们提出利用节点—邻居对扩展正样本对集合。随后,我们引入一个交叉注意力模块来预测邻居相对于锚点节点的支持度分数。该分数量化来自每个相邻节点的正样本支持,并被编码到训练目标中。因此,我们的方法缓解了由负样本和含噪正样本造成的类别碰撞,同时增强了类内紧凑性。我们在五个基准数据集以及节点分类、节点聚类和节点相似性搜索三项下游任务上进行了大量实验。结果表明,我们的方法生成了具有更强类内紧凑性的节点表示,并取得了最先进性能。

英文摘要:

Contrastive learning is a significant paradigm in graph self-supervised learning. However, it requires negative samples to prevent model collapse and learn discriminative representations. These negative samples inevitably lead to heavy computation, memory overhead and class collision, compromising the representation learning. Recent studies present that methods obviating negative samples can attain competitive performance and scalability enhancements, exemplified by bootstrapped graph latents (BGRL). However, BGRL neglects the inherent graph homophily, which provides valuable insights into underlying positive pairs. Our motivation arises from the observation that subtly introducing a few ground-truth positive pairs significantly improves BGRL. Although we can't obtain ground-truth positive pairs without labels under the self-supervised setting, edges in the graph can reflect noisy positive pairs, i.e., neighboring nodes often share the same label. Therefore, we propose to expand the positive pair set with node-neighbor pairs. Subsequently, we introduce a cross-attention module to predict the supportiveness score of a neighbor with respect to the anchor node. This score quantifies the positive support from each neighboring node, and is encoded into the training objective. Consequently, our method mitigates class collision from negative and noisy positive samples, concurrently enhancing intra-class compactness. Extensive experiments are conducted on five benchmark datasets and three downstream task node classification, node clustering, and node similarity search. The results demonstrate that our method generates node representations with enhanced intra-class compactness and achieves state-of-the-art performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑