All in One: Multi-task Prompting for Graph Neural Networks
The Chinese University of Hong Kong(香港中文大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Southeast University(东南大学);Purple Mountain Laboratories(紫金山实验室);Tongji University(同济大学)
Xiangguo Sun, Hong Cheng, Jia Li, Bo Liu, Jihong Guan
备注:
KDD 23 Best Research Paper Award, which is the first for Hong Kong and Mainland China. A Python Library is released as ProG: https://github.com/sheldonresearch/ProG Submitted to SIGKDD'23 in 03 Feb 2023; Receive Acceptance in 17 May 2023 (Rating 3 4 4 4); Submit to arXiv 1st time in 4 Jul 2023
Recently, ''pre-training and fine-tuning'' has been adopted as a standard workflow for many graph tasks since it can take general graph knowledge to relieve the lack of graph annotations from each application. However, graph tasks with node level, edge level, and graph level are far diversified, making the pre-training pretext often incompatible with these multiple tasks. This gap may even cause a ''negative transfer'' to the specific application, leading to poor results. Inspired by the prompt learning in natural language processing (NLP), which has presented significant effectiveness in leveraging prior knowledge for various NLP tasks, we study the prompting topic for graphs with the motivation of filling the gap between pre-trained models and various graph tasks. In this paper, we propose a novel multi-task prompting method for graph models. Specifically, we first unify the format of graph prompts and language prompts with the prompt token, token structure, and inserting pattern. In this way, the prompting idea from NLP can be seamlessly introduced to the graph area. Then, to further narrow the gap between various graph tasks and state-of-the-art pre-training strategies, we further study the task space of various graph applications and reformulate downstream problems to the graph-level task. Afterward, we introduce meta-learning to efficiently learn a better initialization for the multi-task prompt of graphs so that our prompting framework can be more reliable and general for different tasks. We conduct extensive experiments, results from which demonstrate the superiority of our method.
All in One: Multi-Task Prompting for Graph Neural Networks (Extended Abstract)
The Chinese University of Hong Kong(香港中文大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Southeast University(东南大学);Tongji University(同济大学)
Unified Graph Prompt Learning via Low-Rank Graph Message Prompting
School of Computer Science and Technology, Anhui University(安徽大学计算机科学与技术学院);State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology, School of Computer Science and Technology, Anhui University(安徽大学计算机科学与技术学院光电信息获取与防护技术国家重点实验室)
Soft-Prompting with Graph-of-Thought for Multi-modal Representation Learning
School of Computer Science, Wuhan University(武汉大学计算机学院);National Engineering Research Center for Multimedia Software(国家多媒体软件工程技术研究中心);School of Electronic Information Engineering, Henan Polytechnic Institute(河南工业职业技术学院电子信息工程学院);JD Explore Academy(京东探索研究院)
Replay-and-Forget-Free Graph Class-Incremental Learning: A Task Profiling and Prompting Approach
AAII, University of Technology Sydney(澳大利亚悉尼科技大学AAII);School of Computing and Information Systems, Singapore Management University(新加坡管理大学计算与信息系统学院);Department of Computer Science, University of Illinois at Chicago(美国伊利诺伊大学芝加哥分校计算机科学系)
Self-Pro: A Self-Prompt and Tuning Framework for Graph Neural Networks
School of Data Science and Engineering, East China Normal University(华东师范大学数据科学与工程学院);School of Computer and Information Engineering, Shanghai Polytechnic University(上海第二工业大学计算机与信息工程学院)
Generalized Graph Prompt: Toward a Unification of Pre-Training and Downstream Tasks on Graphs
University of Science and Technology of China(中国科学技术大学);Singapore Management University(新加坡管理大学);National University of Singapore(新加坡国立大学);Tecent AI(腾讯人工智能)
Department of Computer Science and Technology, University of Cambridge(剑桥大学计算机科学与技术系);Department of Mechanical and Process Engineering, ETH Zurich(苏黎世联邦理工学院机械与过程工程系)