arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2106.12484cs.LG
收藏

从典型相关分析到自监督图神经网络

From Canonical Correlation Analysis to Self-supervised Graph Neural Networks

University of Illinois at Chicago(伊利诺伊大学芝加哥分校);Shanghai Jiao Tong University(上海交通大学);AWS Shanghai AI Lab(AWS上海人工智能实验室)

Hengrui Zhang, Qitian Wu, Junchi Yan, David Wipf, Philip S. Yu

摘要:

我们引入了一个概念上简单而有效的模型,用于图形数据的自监督表示学习。它遵循前面的方法,通过数据增强生成输入图的两个视图。然而,与专注于实例级判别的对比方法不同,我们优化了受经典典型相关分析启发的创新特征级目标。与其他作品相比,我们的方法不需要参数化的互信息估计,额外的投影仪,不对称的结构,最重要的是,负样本,这可能是昂贵的。我们表明,新目标本质上1)旨在通过学习不变表示来丢弃增强变量信息,2)可以通过在不同维度上解相关特征来防止退化解决方案。我们的理论分析进一步提供了一个新的目标,可以等效地看作是一个实例的信息瓶颈原则下的自我监督设置的理解。尽管它很简单,我们的方法在七个公共图数据集上表现得很有竞争力。该代码可从以下网址获得:https://github.com/hengruizhang98/CCA-SSG。

英文摘要:

We introduce a conceptually simple yet effective model for self-supervised representation learning with graph data. It follows the previous methods that generate two views of an input graph through data augmentation. However, unlike contrastive methods that focus on instance-level discrimination, we optimize an innovative feature-level objective inspired by classical Canonical Correlation Analysis. Compared with other works, our approach requires none of the parameterized mutual information estimator, additional projector, asymmetric structures, and most importantly, negative samples which can be costly. We show that the new objective essentially 1) aims at discarding augmentation-variant information by learning invariant representations, and 2) can prevent degenerated solutions by decorrelating features in different dimensions. Our theoretical analysis further provides an understanding for the new objective which can be equivalently seen as an instantiation of the Information Bottleneck Principle under the self-supervised setting. Despite its simplicity, our method performs competitively on seven public graph datasets. The code is available at: https://github.com/hengruizhang98/CCA-SSG.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑