arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2307.01504cs.SIcs.AIcs.LG
收藏

All in One:面向图神经网络的多任务提示

All in One: Multi-task Prompting for Graph Neural Networks

The Chinese University of Hong Kong(香港中文大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Southeast University(东南大学);Purple Mountain Laboratories(紫金山实验室);Tongji University(同济大学)

Xiangguo Sun, Hong Cheng, Jia Li, Bo Liu, Jihong Guan

AI总结:

针对图预训练与节点、边、图级下游任务不兼容的问题,本文统一图提示与语言提示格式,重构任务空间并结合元学习提出多任务提示框架,实验验证其优越性。

摘要:

近来,“预训练与微调”已被许多图任务采用为标准工作流程,因为它能够利用通用图知识来缓解各应用中图标注不足的问题。然而,节点级、边级和图级的图任务差异很大,导致预训练的代理任务往往与这些多样化任务不兼容。这种差距甚至可能对特定应用造成“负迁移”,从而产生较差的结果。受自然语言处理(NLP)中提示学习的启发——该方法在利用先验知识处理多种 NLP 任务方面展现出显著效果——我们研究图的提示问题,动机是弥合预训练模型与各类图任务之间的差距。本文提出一种新颖的图模型多任务提示方法。具体而言,我们首先从提示 token、token 结构和插入模式三个方面统一图提示与语言提示的格式。通过这种方式,来自 NLP 的提示思想可以被无缝引入图领域。随后,为进一步缩小各类图任务与最先进预训练策略之间的差距,我们进一步研究了多种图应用的任务空间,并将下游问题重新表述为图级任务。之后,我们引入元学习,以高效地为图的多任务提示学习更好的初始化,从而使我们的提示框架在不同任务上更加可靠和通用。我们开展了大量实验,实验结果证明了本方法的优越性。

英文摘要:

Recently, ''pre-training and fine-tuning'' has been adopted as a standard workflow for many graph tasks since it can take general graph knowledge to relieve the lack of graph annotations from each application. However, graph tasks with node level, edge level, and graph level are far diversified, making the pre-training pretext often incompatible with these multiple tasks. This gap may even cause a ''negative transfer'' to the specific application, leading to poor results. Inspired by the prompt learning in natural language processing (NLP), which has presented significant effectiveness in leveraging prior knowledge for various NLP tasks, we study the prompting topic for graphs with the motivation of filling the gap between pre-trained models and various graph tasks. In this paper, we propose a novel multi-task prompting method for graph models. Specifically, we first unify the format of graph prompts and language prompts with the prompt token, token structure, and inserting pattern. In this way, the prompting idea from NLP can be seamlessly introduced to the graph area. Then, to further narrow the gap between various graph tasks and state-of-the-art pre-training strategies, we further study the task space of various graph applications and reformulate downstream problems to the graph-level task. Afterward, we introduce meta-learning to efficiently learn a better initialization for the multi-task prompt of graphs so that our prompting framework can be more reliable and general for different tasks. We conduct extensive experiments, results from which demonstrate the superiority of our method.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑