arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2006.10141cs.LGstat.ML
收藏

图上的自监督学习:深度洞察和新方向

Self-supervised Learning on Graphs: Deep Insights and New Direction

Michigan State University(密歇根州立大学);The Pennsylvania State University(宾夕法尼亚州立大学);TAL Education Group(好未来教育集团)

Wei Jin, Tyler Derr, Haochen Liu, Yiqi Wang, Suhang Wang, Zitao Liu, Jiliang Tang

摘要:

众所周知,深度学习的成功需要大量昂贵的注释数据。这导致了自监督学习(SSL)的发展,其目的是通过在未标记的数据上创建特定领域的借口任务来减轻这种限制。同时,人们越来越感兴趣的是将深度学习以图神经网络(GNN)的形式推广到图域。GNN可以通过简单的邻域聚合自然地利用未标记的节点,而这种聚合无法完全利用未标记的节点。因此,我们试图利用SSL为GNN充分利用未标记的数据。与图像和文本域中的数据实例不同,图中的节点呈现唯一的结构信息,并且它们固有地链接,指示不独立且同分布(或i.i.d.)。这种复杂性对于图上的SSL来说是一把双刃剑。一方面,它决定了采用从图像和文本域到图形的解决方案是具有挑战性的,并且需要专门的努力。另一方面,它提供了丰富的信息,使我们能够从各种角度构建SSL。因此,在本文中,我们首先通过实证研究图上的许多基本SSL借口任务,加深了我们对SSL何时,为什么以及哪些策略与GNN一起工作的理解。受实证研究的深刻见解的启发,我们提出了一个新的方向SelfTask,以构建能够在各种真实世界数据集上实现最先进性能的高级借口任务。重现我们结果的具体实验设置可在\url{https://github.com/ChandlerBang/SelfTask-GNN}中找到。

英文摘要:

The success of deep learning notoriously requires larger amounts of costly annotated data. This has led to the development of self-supervised learning (SSL) that aims to alleviate this limitation by creating domain specific pretext tasks on unlabeled data. Simultaneously, there are increasing interests in generalizing deep learning to the graph domain in the form of graph neural networks (GNNs). GNNs can naturally utilize unlabeled nodes through the simple neighborhood aggregation that is unable to thoroughly make use of unlabeled nodes. Thus, we seek to harness SSL for GNNs to fully exploit the unlabeled data. Different from data instances in the image and text domains, nodes in graphs present unique structure information and they are inherently linked indicating not independent and identically distributed (or i.i.d.). Such complexity is a double-edged sword for SSL on graphs. On the one hand, it determines that it is challenging to adopt solutions from the image and text domains to graphs and dedicated efforts are desired. On the other hand, it provides rich information that enables us to build SSL from a variety of perspectives. Thus, in this paper, we first deepen our understandings on when, why, and which strategies of SSL work with GNNs by empirically studying numerous basic SSL pretext tasks on graphs. Inspired by deep insights from the empirical studies, we propose a new direction SelfTask to build advanced pretext tasks that are able to achieve state-of-the-art performance on various real-world datasets. The specific experimental settings to reproduce our results can be found in \url{https://github.com/ChandlerBang/SelfTask-GNN}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑