arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2302.04626cs.LG
收藏

基于节点-邻域对齐的自监督节点表示学习

Self-Supervised Node Representation Learning via Node-to-Neighbourhood Alignment

University of Wollongong(伍伦贡大学)

Wei Dong, Dawei Yan, Peng Wang

摘要:

自监督节点表示学习的目的是从与监督对应物相竞争的未标记图中学习节点表示。学习信息节点表示的关键在于如何有效地从图结构中获取上下文信息。在这项工作中,我们提出了简单而有效的自我监督节点表示学习通过对齐隐藏表示的节点和他们的邻居。我们的第一个想法实现这样的节点到邻域对齐直接最大化的互信息之间的表示,我们从理论上证明,起到了图形平滑的作用。我们的框架是通过一个替代对比损失优化和拓扑感知的积极采样(TAPS)策略,提出了考虑节点之间的结构依赖性,这使得离线积极选择的样本。考虑到对比学习的内存开销过大,我们进一步提出了一个负自由的解决方案,其中的主要贡献是一个图形信号解相关(GSD)约束,以避免表示坍塌和过度平滑。GSD约束统一了一些现有的约束,并可用于派生新的实现来对抗表示坍塌。通过在简单的基于MLP的节点表示编码器上应用我们的方法,我们学习了节点表示,这些节点表示在一组从小到大的图结构数据集上实现了有希望的节点分类性能。

英文摘要:

Self-supervised node representation learning aims to learn node representations from unlabelled graphs that rival the supervised counterparts. The key towards learning informative node representations lies in how to effectively gain contextual information from the graph structure. In this work, we present simple-yet-effective self-supervised node representation learning via aligning the hidden representations of nodes and their neighbourhood. Our first idea achieves such node-to-neighbourhood alignment by directly maximizing the mutual information between their representations, which, we prove theoretically, plays the role of graph smoothing. Our framework is optimized via a surrogate contrastive loss and a Topology-Aware Positive Sampling (TAPS) strategy is proposed to sample positives by considering the structural dependencies between nodes, which enables offline positive selection. Considering the excessive memory overheads of contrastive learning, we further propose a negative-free solution, where the main contribution is a Graph Signal Decorrelation (GSD) constraint to avoid representation collapse and over-smoothing. The GSD constraint unifies some of the existing constraints and can be used to derive new implementations to combat representation collapse. By applying our methods on top of simple MLP-based node representation encoders, we learn node representations that achieve promising node classification performance on a set of graph-structured datasets from small- to large-scale.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑