arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2411.01157cs.LG
收藏

无负样本的图自监督高斯嵌入

Negative-Free Self-Supervised Gaussian Embedding of Graphs

State Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室)

Yunhui Liu, Tieke He, Tao Zheng, Jianhua Zhao

AI总结:

针对现有图对比学习依赖负样本导致计算与内存开销高的问题,提出无负样本的高斯嵌入方法,通过对齐表示分布与各向同性高斯分布实现均匀性,在7个基准上以更低成本取得了有竞争力的性能。

摘要:

图对比学习(GCL)是近年来兴起的一类颇具前景的图自监督学习框架,可在无标签条件下学习具有判别性的节点表示。GCL 广泛采用的目标函数具备两个关键特性:对齐(alignment)与均匀性(uniformity),前者使正样本节点对的表示相互对齐,后者则将所有表示均匀分布在超球面上。均匀性在防止表示坍缩中起着关键作用,其实现方式是推远不同节点的增强视图(即负样本对)。因此,现有 GCL 方法本质上依赖于提升负样本的数量与质量,这会带来高昂的计算需求、内存开销,还可能引发类别碰撞问题。本研究提出一种无负样本的目标函数来实现均匀性,其灵感来源于:服从归一化各向同性高斯分布的点会均匀散布在单位超球面上。因此,我们可以最小化学到的表示分布与各向同性高斯分布之间的距离,以提升节点表示的均匀性。我们的方法与其他方法的区别还在于,它无需参数化互信息估计器、额外的投影头、非对称结构,且最关键的是无需负样本。在7个图基准数据集上的大量实验表明,与现有 GCL 方法相比,本文所提方法以更少的参数、更短的训练时间和更低的内存消耗,取得了具有竞争力的性能。

英文摘要:

Graph Contrastive Learning (GCL) has recently emerged as a promising graph self-supervised learning framework for learning discriminative node representations without labels. The widely adopted objective function of GCL benefits from two key properties: \emph{alignment} and \emph{uniformity}, which align representations of positive node pairs while uniformly distributing all representations on the hypersphere. The uniformity property plays a critical role in preventing representation collapse and is achieved by pushing apart augmented views of different nodes (negative pairs). As such, existing GCL methods inherently rely on increasing the quantity and quality of negative samples, resulting in heavy computational demands, memory overhead, and potential class collision issues. In this study, we propose a negative-free objective to achieve uniformity, inspired by the fact that points distributed according to a normalized isotropic Gaussian are uniformly spread across the unit hypersphere. Therefore, we can minimize the distance between the distribution of learned representations and the isotropic Gaussian distribution to promote the uniformity of node representations. Our method also distinguishes itself from other approaches by eliminating the need for a parameterized mutual information estimator, an additional projector, asymmetric structures, and, crucially, negative samples. Extensive experiments over seven graph benchmarks demonstrate that our proposal achieves competitive performance with fewer parameters, shorter training times, and lower memory consumption compared to existing GCL methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑